Sterling P. Newberry
CBI Labs, 11 S. Westcott Road, Schenectady, NY, 12306, USA
Abstract. This progress report describes problems in introduction of
"Morphing" and 3-D techniques into Time-Lapse X-ray microscopy. The issues faced, apply to imaging with any form of ionizing radiation at any magnification level. Some historical background is given.
1.1 Origin of Time-Lapse
By remarkable coincidence, just two years before the discovery of X-RAYS, Ludwig
Mach, son of Ernst Mach, demonstrated visual acceleration of a slowly occurring event. In this first TIME-LAPSE demonstration he speeded up the growth of plants.
This was preceded by eighteenth century artists displaying moving images long before the discovery of photography. After the availability of photography, but nearly a half century before the discovery of X-rays, the idea of time alteration by photographically freezing rapid motion was pursued. But the only thought was to slow down action so that the observer could study the event. Ernst Mach was one of the leaders in these studies because of his interest in shock waves. A quarter century elapsed between spark photography to slow down action and work to speed up slow events was accomplished. Ernst Mach was a visionary who suggested many innovations, some of which we have yet to seriously consider. He suggested time compression, our form of time-lapse, which his son published in 1893
[
1, 2
]
.
Fig. 1. Single frame camera. Left on microscope, right inverted to show specimen and target relations
1.2 Early X-Ray Microscope Time-Lapse Work
Time-lapse as our generation first learned about it, was a moving picture technique to speed up very slowly moving events in order to appreciate the dynamics of the event, such as the opening of a flower blossom. It was very simple. A movie camera was set

I - 176 S. P. Newberry up to take a short burst of frames at specified large intervals of time. After image reversal development, this film was then played back at normal projection rates. In
1956 after the author had increased the spot intensity ten fold on our General Electric version of the Point Projection Shadow X-ray Microscope, he suggested that we could now take low magnification exposures at the rate of 3 to 5 frames per second and could therefore be able to make useful time-lapse recordings. Our group mounted a small 16 mm film projector in place of the film camera. We produced time lapse runs of living, diminutive species of insects and spiders and of metal corrosion which were shown at the Electron Microscopy Society of America (EMSA) meetings. Later at the
Los Angeles meeting of EMSA in 1958, we showed 3D time-lapse X-ray Microscope sequences. The early onset of radiation sickness in the living specimens was obvious, but not important at the time and radiation damage to non-living specimens was not detected at those magnification levels.
1.3 Current Time-Lapse Programs Initiated
Interest in Time-Lapse X-ray microscopy was revived among my current collaborators in the spring of 1989 when I was approached by Dr. Vozzo, of the US Dept. Of
Forestry, to make higher resolution radiographs of Black Walnut tree fruit (Juglans
nigra). Together we began a cautious study of the possibility of making meaningful studies of seed germination by X-ray time-lapse, since the seeds might become quite sensitive to radiation during germination. There were many surprises along the way.
The first was the finding that laboratory use of motion picture film was no longer convenient to the point of being impractical. The next was the discovery that all electronic photography was equally unattractive and inordinately expensive. The problems of how to induce the onset and completion of germination in a microscope have also been very challenging. Once the seed has been imbibed water, the contrast is very low. At this stage computer enhancement of the recorded images is essential. In
Many of the applications, one wants to make of time-lapse, genetic changes may or may not be of concern, but other radiation damage is always a possibility and it is therefore emphasised in this report. Another area addressed is how to make time-lapse more objective.
2.1 Limiting Radiation Damage
When imaging with ionizing radiation (I.R.) the primary goal is to use as few ionizing photons as necessary to get the desired information. Some of the ways in which radiation can be wastefully applied are:
1) Using I.R. to set up the instrument with specimen near by or in place.
2) Searching and orienting the specimen by I.R.
3) Magnification, i.e. pixel resolution, more than needed.
4) Illumination beyond the field of view required.
5) Inappropriate choice of radiation band width and frequency distribution.

Progress Report on Time-Lapse X-Ray Microscopy I - 177
In time-lapse imaging with ionizing radiation the major ways to minimize the radiation burden are:
1) Use a stable instrument which can be adjusted prior to inserting specimen.
2) Orient and position the specimen with non I.R. or mechanical positioning.
3) Do trial runs on similar specimens to determine rate of change vs time.
4) Strive to make exposures only after a significant change has occurred.
This last point, to strive to make each succeeding exposure only after a significant change has occurred in the specimen or event, is best served by being able to examine each exposure before the next one is due.
2.2 "Morphing" a Step Toward Objectivity
Morphing is a kind of virtual reality, in which one acquired image is transformed, by creation of virtual intermediate steps, into a different acquired image. It was developed for applications where the starting and final images are quite different such as transforming one person’s face into another’s or even into an animals face. This had previously been done for television and cinema by photographing the work of makeup artist and then "Lap-Dissolving" the photographs of the series of makeup artists models into a continuous movie. The Lap-Dissolve Technique simply superimposes the two adjacent images and then ramps the intensity from 100 % of the first to 100 % of the second. The advantage of morphing lies in the fact that each step of transposing between adjacent images, moves each pixel a proportional step toward its new position and new intensity. Thus when creating a movie by morphing, each frame is a sharp, measurable image whereas the intermediate images with Lap-Dissolve are blurred. As we propose to use the Morphing Technique for time-lapse, we will substitute intermediate images which will be real pictures for the computer generated images usually employed. Now the adjacent images are of an actual subject which has changed in the real world by a small step. Thus the fill in frames should be, to a first approximation, what one would have recorded if the "real exposure" had been made at the virtual reality time interval. Here morphing could, in theory, permit objective rate of change measurement in microscopic areas. The Time-Lapse Movies we showed at the prior conferences
[
3, 4
]
used the Lap-Dissolve Technique. In order to obtain chrisper images and objective measurement, we are presently committed to development of the Morphing Technique. The Advanced Microscopy and Imaging
Laboratory (AMIL)
[
5, 6
]
has developed Morphing Software suited to time-lapse needs. In Living specimens our major problem is to develop methods to easily determine corresponding locations in adjacent images.
2.3 Three Dimensional Techniques a Desirable Extension
An important advantage of X-rays for 2D imaging is that their absorption is quantitatively related to an object’s chemical identity so long as the object is isotropic in the path of the imaging ray. X-rays are also able to provide superior penetration and depth of focus, but this quality is usually inconsistent with an isotropic path for the individual imaging ray. Thus when spectroscopic information is desired, there is no advantage in using X-rays over light or electron microscopy for two dimensional images except for large opaque objects. It is well known that by tomography a thin

I - 178 S. P. Newberry section of an object can be constructed to which absorption analysis can be applied. It is difficult to combine tomography with X-ray microscopy as evidenced by the substantial number of papers in this conference reporting work toward this goal.
Tomography is expected to have limited use in time-lapse because of the very large number of exposures and the substantial data handling required. The author has proposed
[
8
]
the use of 3D aerial mapping techniques to isolate the absorption of a small vertical extent of the imaging ray path and thus add false color or other spectroscopy to time-lapse. Improvements in tomography at AMIL has provided algorithms which should make this approach practicable.
3.1 Bridging the Gulf Between Specimen and Observer’s Needs
We have emphasized the limited number of exposures which are permitted to be taken of the specimen because of potential radiation damage. We must also consider the large number of motion picture frames which the human observer needs to fully comprehend the dynamics of the time-lapse presentation. For the space between successive exposures of the object we should insert a full two seconds of viewing time. This can mean as many as 128 computer generated frames to fill two seconds of moving picture time for the benefit of the human observer. Thus, while we may use manual methods for computer enhancement of the original exposures, we must automate the generation of the fill in frames. Except for capturing the original exposure, we have converted to all digital imaging manipulation. Computers with processing power to handle our work are now available to university and industrial personal. We are now able to enhance the original exposures with a Power MAC.
Ultimately we can expect personal computers to be adequate for the automation of the fill in frames and the movie production.
3.2 The Hybrid System We Employ
Currently, Photography is much better for our data collection, providing of course we select the magnification and exposure to just give the required information, thus keeping the radiation exposure down. We answer the requirement to check each exposure before the next one by putting a short length of "Piggy-back" film behind the roll film and retrieving it in the dark room for quick development. At 10 Kev and above, the exposures appear identical in quality. Use of photography for data collection permits use of very large, off site computers for the movie preparation.
After twenty years of anticipating an all electronic system for the Shadow Projection microscope, from image capture to presentation, we still find it necessary to use this hybrid system.
The goal of an all electronic system is known to be possible, and believed to be near at hand. Even with photographic image capture we are now committed to digital image manipulation and movie production. We are finding many difficulties to be overcome in working with seed germination studies under this commitment.

Progress Report on Time-Lapse X-Ray Microscopy I - 179
3.3 Equipment
Even though a separate program has shown that a modernization of the Shadow X-ray
Microscope is desirable in order to make it widely available, all of our work has been, and still is, done on the 1956 vintage microscope
[
7
]
. Fig. 1, showing the Olympus
Pen film back, does not show the lead cover which would hide the camera. This half frame camera (minus lens) permits very close camera distance and twice the number of exposures per roll. For all cameras a lead cover must be used for safety of personal.
We are pleased with the performance of Kodak Tmax film of 100 ASA rating developed in D-76 full strength or equivalent. We can use any roll camera with lens removed or some roll film backs. A focal plane shutter on bulb setting is preferred since it does away with need for a thin, light tight filter..
3.4 Auxiliary equipment
Most applications require special specimen holders and environmental controls. In the germination of seeds, such as the garden pea visible in fig. 1, provision must be made for taking the setup apart and recentering because geotropism requires that the seed be kept, between exposures, in a separate humidity controlled vessel at right angles to its position in the microscope. Simple kinematic design is required to relocate the specimen for each exposure.
We are working both in hardware and software to solve the following problems:
1) Living objects change their location making it hard to follow where the specimen structure, on a voxel level, is with respect to the image pixels. One possibility is to skeletonizing a few image bright points and following their motion with respect to the source location, which is very stable. Source location can be identified using two stationary markers in the border of the field. Another possibility is to make micro chip markers and glue one at each end of the specimen with crazy glue.
2) We have been experimenting with artificial soils for study of root interaction with soil. This work continues.
3) Another major area, is in efforts to take advantage the rapid developments in personal computers, as they occur, to make data reduction and morphing practical on site.
The most useful next step from the observer’s stand point is the revival of 3D timelapse. Taking 3D images is easy. One merely makes two exposures at each interval and moves the specimen a small, repeatable amount between the two exposures or conversely moves the electron beam a much smaller amount as we did in the original
3D work. The problem is that the special, mechanical 3D movie projectors employing a single reel of film are no longer available. Synchronizing two separate projectors is very difficult and is not suitable for individual or small group viewing. To make 3D

I - 180 S. P. Newberry time-lapse practical we should develop a personal computer system which can play our movies from a single tape or CD. With the advent of digital TV, we can expect 3D video tape players to become available also.
1 J. M. Eder, History of Photography, Translated by E. Epstean (Columbia U. Press,
1945)
2 L. Mach, Phot. Rundschau, p 121 (1893)
3 S. P. Newberry, Time-Lapse X-Ray Imaging by Shadow Projection Microscopy,
Proceedings XRM ’90 London, p 367 (vol. 67 of Springer Series in Optical
Sciences, A. G. Michette, G. R. Morrison and C. J. Buckley, Eds.)
4 J. A. Vozzo, S. P. Newberry and M. Marko, Time-Lapse X-Ray Microscopy
Video of Seed Germination, p 230, XRM ’93 Moscow, V. V. Aristov and
A. I. Erko Eds., Bogorodskii Pchatnik Publishing Co., Chernogolovka,
Moscow region, Russia
5 The AMIL ("Advanced Microscopy and Imaging Laboratory") is an interdepartmental facility at SUNY in Buffalo, NY, 14260, USA; headed by
Professor P. C. Cheng
6 P. C. Cheng, et al. This conference
7 S. P. Newberry, The Shadow Projection Type X-Ray Microscope, p 126, in X-Ray
Microscopy, P. C. Cheng and G. J. Jan Eds., Springer (1987)
8 S. P. Newberry in Focus on Microscopy ’95, to be published by World Scientific,
P. C. Cheng et al. Eds.