<meta name="Keywords" content="film restoration; film preservation; preserve; films; Warner
Bros.; cinerama; digital imaging; color; red blur; flattening; registration; flicker; qian lin; HP Labs;
HP Lab; Hewlett-Packard; Hewlett-Packard Laboratories; lab; laboratories; R&D; research;
research & development; invent; labs.">
<meta name="Description" content="Although classic films are easily damaged with age, few
have been restored because the process has been both painstaking and costly. Now HP and
Warner Bros. researchers are automating the process and yielding some spectacular results.">
Pull quote: Classic films are fragile treasures, but a new system that automates film restoration
could preserve them for future generations to enjoy.
Right navigation:
>> Imaging systems research /research/isl/
>> Warner Brothers http://www2.warnerbros.com/main/homepage/homepage.html
Movie makeover: HP and Warner Bros. give old movies new life
By Simon Firth
It’s that big scratch across Scarlett O’Hara’s face; the red blur obscuring Robin Hood’s heroics; or
the white blotches that pepper King Kong as he climbs the Empire State Building.
Classic movies are fragile treasures. They pick up dust, are easily scratched, and often shrink or
fade and get otherwise disfigured as they age.
The most famous will occasionally be painstakingly restored by hand, says HP Labs’ Qian Lin.
But of the thousands of other movies sitting in archives around the world, Lin says, “very few go
through this manual processing, because the cost is very, very high.”
That’s a big deal to Hollywood studios like Warner Bros., owners of Gone With the Wind, Robin
Hood, King Kong and thousands of other classic features. pls note italics in film titles
HP researchers, in collaboration with Warner Bros. Motion Picture Imaging, have come up with a
process for automating much of the restoration process and are now looking to bundle their set of
media-processing techniques into a single engine, or video-processing pipeline. It’s already
yielding some spectacular results.
In test clips Scarlett now sweeps down the stairs to meet Rhett unaccompanied by background
glitches; Robin Hood fights his battles in glorious, sharp Technicolor; and King Kong climbs
through a clear Manhattan sky.
“We have many of the most beloved and important films ever made in our library, and we feel a
strong responsibility to preserve these films so that future generations will be able to enjoy them,”
says Chuck Dages, executive vice president of Emerging Technology at Warner Bros. “And from
a pure business standpoint, these films are an invaluable asset and need to be properly cared
for."
Automating clean-up
Studios need to re-copy their movies every time a new distribution technology comes along, says
Dages.
Following VHS video and then DVD, the industry is now moving toward the next generation of
optical disc formats: High Definition DVD (HD-DVD) and Blu Ray formats. The new formats offer
studios a great economic opportunity, Dages says, but a challenge too.
“Older movies that have been transferred to video,” he explains, “just don’t cut it when they go to
the high-definition television formats”
In response, studios are making super-high resolution digital transfers of each frame of many of
their library movies. But without special processing, whatever dirt, scratches or other
imperfections that mar the original frames are transferred to the digital copies; all in high
definition.
Until now the only way to get rid of imperfections, explains Dages, was to have “someone sit with
a mouse and laboriously click to take the dirt out.” No wonder, he says, Warner Bros. was hoping
“that there was HP technology that could be applied to help automate the process.”
Building on experience
Although HP Labs had never looked at film restoration, researchers had plenty of experience
working with still photography and personal video.
“We found that a lot of that expertise can be applied to the film restoration process,” says Lin,
who leads the research team.
Researchers were able to apply much of what they'd learned from an earlier investigation of
color-sensor technology to address blurring of images in older color library titles. Work on singlelens cameras that de-blurred an area of color using edge information from other patches of color
in the frame also proved useful, as did research into video super-resolution, a technique for
combining information from nearby frames to increase a particular frame’s resolution.
How it works
The automated restoration process works like this: Each film clip is first run through a set of
algorithms that selects out only the frames likely to contain an image artifact. For example, says
researcher Amnon Silverstein, “we look for a pixel that changes color dramatically from frame to
frame. Or within a frame we look for a small area that is very different from everything else in its
neighborhood. And in color, we look for things that are saturated and unique.”
A second pass fixes many of the glitches. When dust creates a bright red, green or blue spot in
the frame, for instance, the researchers' software fills it in with information interpolated from the
area around it.
Although some particularly complicated frames must still be repaired by hand, automated fixes
dramatically reduce the amount of manual work required.
In the cases where the team had access to scans of the original set of color negatives, the
images took on a new life.
“Thanks to this technique, you can now look at the chain link in Robin Hood’s armor or the lace in
Scarlett O’Hara’s dress and they pop out with incredible detail and a purity of image that you
never saw before,” says Richard Place, global manager for all of HP’s Time-Warner accounts.
Different kinds of film offer different challenges
Digitized frames from black and white movies, for example, contain less information, making
scratches, dust and other errors harder to spot. Early black and white film also tends to look
grainy and suffer from flicker, a result of the poor shutter control on early cameras, giving each
frame a different exposure.
Perhaps the team’s biggest challenge was Cinerama. Only 13 features were made using this
widescreen technique – a format that requires a cinema equipped with a curved screen and three
projectors that run simultaneously – but nearly all are considered classics.
The HP team worked on clips from the 1962 epic, How the West Was Won. Beyond the usual dirt
and scratches, Cinerama films doubly distort when transferred to home-viewing formats. Because
they were projected onto a curved screen, they look odd when shown on a flat screen. And they
suffer from clear overlaps and lines where the three images join.
“To resolve the vertical-line issue, the team borrowed technology for putting ink on paper, which
is very similar to putting ink on film,” reports Richard Place. “They then used another algorithm to
eliminate the flattening distortion.”
The result, says Warner Bros. executive Dages, “is remarkable. The technology really has turned
what has been an issue for many, many years into a solution that allows us to repurpose the film.”
Speeding up the process
HP and Warner Bros. Motion Picture Imaging envision a process that would either run
automatically or be ‘dialed’ up or down in sensitivity depending on what is happening in a
particular sequence (the more action, the more likely that one of flying arrows, for example, might
be mistaken for a scratch, requiring the software’s sensitivity to be turned up).
Cleaning entire films requires enormous computing resources.
“These films take up terabytes of storage,” says researcher Silverstein. “So we’ve been looking at
how you can take a film, send it out across a set of servers, process it efficiently, and handle
cases when machines are added and removed.”
Each two-hour movie has 172,800 frames – if you're processing that on a single machine at
about five minutes per frame, it would take 600 days to complete an entire movie. By using a
cluster of servers, engineers can speed up the process considerably.
Such a process would employ state-of-the-art data security from HP, a crucial feature for content
owners worried about video piracy.
Film remains archiving choice
It’s not just Hollywood that could benefit from the innovations behind the video-processing
pipeline.
There are major theatrical movie archives in India and Hong Kong, for example, as well as large
research holdings in public collections like the U.S. Library of Congress.
Although digital is gaining ground, film remains the medium of choice for making movies and
high-profile television dramas.
And despite its flaws, says Warner Bros.' Dages, film continues to be the studio’s preferred
archival material. “We know,” he says, “we can put a piece of film in the vault and under proper
conditions, it will last up to 600 years. It is a great archival media."
________________________________________________
Simon Firth is a writer and television producer living in Silicon Valley.