Best Practices for the Digital Conversion of Dynamic Media
Stephen Naron
David Walls
Molly Wheeler
12 August 2008
Table of Contents
Best Practices for the Digital Conversion of Dynamic Media ......................................3
Copyright
Digitization principles
Audio...................................................................................................................................6
Section 1: Workflow best practices ............................................................................ 6
Costs
Requests for Information
Requests for Proposals
Section 2: Technical specifications ..............................................................................7
Metadata
Quality Assurance and Quality Control
Future Directions
Section 3: Additional Information ...............................................................................9
Vendors we have used
Audio Preservation Organizations
Additional resources
Film ...................................................................................................................................10
Section 1: Workflow ..................................................................................................10
Identification
Assessment and Storage Preparation
Storage
Cataloging metadata
Section 2: Technical specifications ............................................................................12
Reformatting
Preservation formats
Access Options
Section 3: Vendor relations ........................................................................................15
Section 4: Examples of successful projects at Yale or other institutions ...................15
Additional resources ..................................................................................................16
Video ................................................................................................................................17
Section 1: Workflow best practices ...........................................................................17
Condition assessment
Video setup
Metadata capture
Quality control
Section 2: Technical specifications ............................................................................19
Preservation file formats
Access file formats
Digital preservation
Section 3: Vendor relations ........................................................................................21
Section 4: Examples of successful projects at Yale or other institutions ...................22
Additional resources ..................................................................................................22
Summary Recommendations .........................................................................................23
Media Best Practices Task Force Members .................................................................24
12 August 2008
2
Best Practices for the Digital Conversion of Dynamic Media
What was once described as audiovisual collections is now frequently referred to as “dynamic
media.” When electronic media entered the world of art exhibits, the term "dynamic" became
commonplace as a way to distinguish between static art media and to emphasize the sound and
movement of audio, video, and film content.
The sight and sound of film, video, and audio make the past come alive for us in a way that books
and manuscripts cannot. Unlike paper based collections, with life spans of 100 years or more, the
life span of dynamic media is on average only about 50 years. Decay and damage is not easily
visible. All dynamic media requires the intervention of a machine to play back the signal or
display the image at the prescribed rate of speed. Rapidly advancing technology frequently
makes the media format obsolete before the media itself begins to decay.
Aside from large, well known collections such as Historic Sound, or the Film Study Center, much
of the dynamic media within Yale University Library consists of primary-source materials created
by amateur individuals who recorded or filmed an event of interest to them. Much of this media
was acquired by libraries and archives at Yale when collections of personal papers were also
accessioned. The practice for many years was to separate all media from the more highly desired
personal papers and manuscripts. The result of this practice is that media collections have
received scant attention and are largely inaccessible to researchers at Yale due to fragility,
obsolescent playback technology, and a lack of description and cataloging to allow researchers to
discover the items at all.
Due to the fragility and rarity of these primary-source materials, the digitization of dynamic
media should be carefully planned using the consulting services of trained professionals. The
attempt to save money by treating media reformatting as a “do it yourself” project often results in
damage to original materials requiring expensive restoration; if indeed, the item can be restored at
all. These best practices are intended to serve as a guide for understanding the principles and
technological issues surrounding the digital reformatting of dynamic media. They are not
intended as a substitute for the advice of professional experts.
For the purposes of this document, reformatting and digitization are essentially interchangeable
terms, except when creating direct film-to-film print transfers. In 2005, Quantegy, the last
remaining manufacturer of analog audio tape, filed for bankruptcy, removing any reliable analog
solution to audio reformatting and forcing audio archives to think of digitization as a means of
both preservation and access. There is also consensus among moving image archivists that video
reformatting, particularly of legacy analog materials, will become increasingly, if not exclusively,
file-based in the immediate future. The National Film Preservation Foundation still requires the
creation of a film-to-film transfer print made on polyester film stock from a direct contact with
the original film. However, the expense of this process, the lack of equipment to playback many
original motion picture image and sound formats, and the increasing user demand for streaming
moving images through websites such as YouTube have placed an emphasis on the digitization of
original films for access.
Most of the reformatting of dynamic media at Yale has been focused on providing access to
researchers who frequently want a .wav file, or DVD of a recording or video for their own
research. This process occurs on an ad-hoc basis and each library or department within the
Library has frequently pursued a different strategy and worked with a variety of vendors to
reformat media items or collections with varying degrees of success. Few projects have had any
preservation focus at all. One main reason for this lack of preservation activity is a reflection of
12 August 2008
3
the technological complexity and costs associated with digitizing, archiving, and serving media
collections. However, the most prevalent reason may simply be a lack of understanding among
collection managers. They may be unaware that not only are there media reformatting best
practices, but also that there is expertise within the Library to write project specifications and
contracts and even manage digitization projects.
The lack of storage space in the form of a trusted digital repository for digital media is also of
concern when planning future digitization projects. A trusted digital repository for media at Yale
would conceivably require enterprise-level storage with built in RAIDs to archive and serve
exabytes of data. Once archived, a diligent program of life-cycle management is critical to ensure
long-term access.
Copyright
Because the digitization of media assumes wide spread access, typically normal assumptions
about fair use do not apply. Copyright for media is complex and the issues surrounding the
digitization of media frequently confound professional legal experts. Simple ownership of the
physical recording, video, or film, even if it is non-commercially produced, does not grant the
owner rights to the intellectual content of that work. Also, a variety of rights belonging to more
than one person may be bound up contractually within the creation of single media item.
However, no interpretation of copyright suggests that a library cannot intervene to prevent a
valuable information resource from becoming unusable due to physical decay. Rights issues
should be considered before items are digitized.
Digitization principles
1. Do No Harm!
All media should be considered fragile. Because the content of media items or collections
is frequently unknown, there is a temptation to play media to discover its content. No
attempt should be made to play a tape, video, or film, which has been stored until its
condition has been assessed by a preservation professional.
2. Consult and Plan First!
Know what the desired outcome of your project is. How do you intend to use the digital
media created in the project? Who is the intended audience? Are there local resources
already available to archive and serve the media formats you want?
12 August 2008
4
3. Scan Once!
Create as many derivatives as you need. A quick scan of a video, phonodisc, or film to
create a digital use copy places just as much wear and tear on the item as a high quality
scan that can be archived for future use.
4. Digital Copies are Surrogates for the Original. They should not be considered
replacements unless the original copy is too fragile to play again! The original media item
should be evaluated, cleaned, and rehoused in an appropriate preservation-grade container.
Most importantly, it should be bibliographically tracked, and stored in an environment that
maximizes its long-term stability.
12 August 2008
5
Audio
David Walls, Preservation Librarian, Head, Reformatting and Media Preservation
A variety of analog formats exist within audio collections at Yale.



Edison Cylinders
Phonodiscs: created from a wide variety of materials
Tape: Reel to reel or cassette
Each type consists of a means of fixing a signal as a continual wave either within the surface of a
recording, such as the grooves on a phonodisc, or as a magnetic signal on a surface of charged
particles, such as that found on tape.
Section 1: Workflow best practices
It is assumed that the digitization of audio will require the services of an outside vendor.
Therefore, it is important to be able to specify exactly the services that the vendor is to perform.
When the goals of the digitization project have been defined and all copyright issues reasonably
resolved, it is best to specify all services and deliverables requested in writing.
Costs
The cost of digitizing audio may vary depending on the length, the condition of the original item,
any restoration or cleanup needed to clarify or restore the analog audio signal, and the variety of
desired output files and their physical delivery.
Requests for Information
These can be sent to vendors when planning an audio digitization project. Describe the item or
collection to be digitized. In some cases, the vendor may want to even physically see a sample of
the audio to be digitized to evaluate its condition and provide a more accurate estimate of the
costs involved. Ask about sampling rates, metadata, and what file formats the vendor typically
delivers.
Requests for Proposals
Proposals are usually written based on earlier responses to requests for information from vendors.
A RFP can be sent to several vendors to evaluate the services they provide and specifically how
they will treat the material you send them to digitize. The RFP should ask the vendor to specify
the quality control procedures they use and how any errors in their work will be resolved. An
RFP should also request that the vendor provide adequate proof of insurance coverage for items
located on their premises as well as reference contacts for recent projects. An RFP can be written
and kept on file by a vendor as a set of specifications for future work received from a client. It
can also be written so that it can be easily reworded to serve as a contract between the vendor and
the library for the specific digitization project. All contracts for services should specify exactly
the deliverables that the vendor is to provide and include a quality control threshold that the
vendor agrees to meet. This statement of quality allows you, the customer, to reject whole
batches of work from the vendor if the quality is not acceptable.
12 August 2008
6
Section 2: Technical specifications
All analog recordings must be played in real time when they are digitized. If a recording plays
for an hour, then the analog signal must be recorded at the normal playback speed.
Audio is digitized by a process known as “sampling.” In sampling, the continuous output of the
analog wave signal is sampled at a bit depth and frequency to sufficiently capture enough
information to render the digital signal an accurate representation of the original.
Typical sampling rates include (24bit, 192kHz), (24bit, 96kHz), (24bit 88.2kHz), and (16bit,
44.1kHz) in descending file size. The sampling rate should be specified according to the level of
information on the original recording. For example, a master recording of an opera may be
recorded at 24, 192 or 24, 96, to capture the full dynamic range of the orchestra and voice.
However, scanning a recording of a simple voice conversation at this level would produce only a
large file, without providing any additional information. For most spoken word recordings, 16,
44.1 is the typical rate. However, there is at present no agreement on a standard sampling rate.
The European Commission sponsored a series of hearings on audio digitization in 2006. There
was some consensus that the best scanning rate was 24bit, 96kHz, but the panel of experts could
not ultimately agree and no standard was formally adopted. The BBC, British Broadcasting
Company, archives its audio programs at a sampling rate of 16 bit, 44.1.
In the digitization of audio, a preservation master .wav file is typically created at a specified
sampling rate. The .wav file format is an uncompressed or lossless audio bit stream format for
storing audio data on both IBM and Apple computer platforms. If any cleanup or enhancement
of the file is needed, the original unmodified .wav file is copied to produce the modified file or
preservation master-intermediate. Both versions of the files are typically archived.
Another preservation master file standard exists called a “Broadcast Wave File” or .bwf. This file
type is essentially an extension of the .wav file. It was designed to allow the seamless
transmission of audio back and forth across various broadcast platforms for radio and television
transmission of sound. A broadcast wave file also contains a basic metadata stream containing
information about the title, origin, date, and time of the content. However, the .bwf file format is
not compatible with all systems that can play standard .wav files. For this reason, audio metadata
for standard .wav files is typically expressed using a standard METS schema as an .xml file.
Combinations of sampling rates can be used to ensure that all of the information in the original
recording is captured and archived. While the .wav file serves as the audio version of the archival
.tiff image format, it is too large to stream or copy onto most audio devices. For this reason, the
original .wav file is derived to produce an .mp3 or RealAudio file for ease of copying onto most
digital audio devices or for providing streaming audio over the web. Both .mp3 and RealAudio
are lossy compression formats and for this reason are not considered to be archival. There are
additional lossy formats for distribution such as .mp4 and Liquid Audio.
Compact discs may also be created from .wav files by down sampling the file to the most
common CD format also known as Redbook Audio. This file format specifies the conversion of
the digital audio bit stream into the pits on the surface of the compact disc that are read by the
laser light or “stylus.” Compact discs should be considered expendable use copies.
12 August 2008
7
Metadata
Most publications about audio metadata say that the vendor should generate as much metadata as
possible about the nature of the original recording and the transfer process. The amount of
technical metadata available may be governed by the existing equipment that the scanning vendor
uses to digitize the original analog copy. However, vendors should also document any anomalies
or problems found in the digitization process such as delamination, splice problems, repairs,
speed variations, blocking, sticky shedding, or significant blank spaces without sound.
Descriptive metadata elements should also include all documentation that accompanies the
original recording. Audio metadata is commonly recorded using the METS schema, although
MODS, and PREMIS, are also used. The schemas are usually recorded as an .xml file.
Quality Assurance and Quality Control
The terms quality control and quality assurance are frequently used interchangeably. However, in
reality, quality assurance is a function of the design of the digitization project itself that ensures
that a quality product will be created in the first place. Quality assurance is built in to the design
specification for the project that are included in the RFP, your evaluation of the vendor’s
proposal, and the final contract that you and the vendor agree to for the services you need.
Quality control consists of the activities designed to verify compliance and quality. It is time
consuming and therefore expensive in terms of the staff time needed to carry it out. Typically a
random sample representing 20% of the total, including accompanying metadata files should be
examined. Random errors are inevitable; however consistent errors of the same type indicate a
poor QC at the vendor and are an issue that should be addressed. The first QC activity is to verify
all data files using Checksums and Jhove (JSTOR/Harvard Object Validation Environment). A
QC on the performance of the audio transfer can only be performed by listening to portions of the
digitized files. There are some newly developed automated tools for checking the quality and
performance of digital audio and their adoption and installation in an in-house facility is likely as
the Reformatting and Media Preservation Program plans to meet the future needs of audio
preservation at Yale.
Future Directions
Paradoxically, expertise in the preservation of analog media formats is literally passing away day
by day at a time when experience suggests that the future of audio reformatting will continue to
be a period of innovation and change. While best practices have emerged, it is important to
understand that no standards exist. All audio digitization projects are a compromise between the
preservation needs of the original object and the access needs of both present and future users of
our collections. Data from a future media survey will provide valuable direction for determining
the preservation and reformatting needs of media collections and will also serve as a guide for the
Reformatting and Media Preservation Program’s response to that need.
12 August 2008
8
Section 3: Additional Information
Vendors we have used
Cutting Corporation www.cuttingarchives.com
Safe Sound Archive www.safesoundarchive.com
The Media Preserve www.mediapreserve.com
Audio Preservation Organizations
ARSC: Association for Recorded Sound Collections www.arsc-audio.org
SAA: Society of American Archivists www.archivists.org
TAPE: Training for Audio Preservation in Europe www.tape-online.net
Additional resources
Capturing Analog Sound for Digital Preservation, CLIR, 2006
http://www.clir.org/pubs/reports/pub137/pub137.pdf
Sound Directions, Digital Preservation and Access for Global Audio Heritage, Indiana
University, NEH funded Sound Directions Project
http://www.dlib.indiana.edu/projects/sounddirections/papersPresent/index.shtml
Walls, David. Saving Sound III, Working with Audio Vendors. 24 June 2007. American Library
Association Conference, Washington, D.C.
12 August 2008
9
Film
Molly Wheeler, Archivist, Beinecke Rare Book and Manuscript Library
There are currently no official best practice guidelines for film in the field of moving image
archives though some institutions have developed their own. These practices are informed by the
same sources (most importantly and commonly the National Film Preservation Foundation
Preservation Guide) and the field eagerly waits for other best practice guidelines to emerge. For
example, the Library of Congress and the Harvard Film Archive are currently developing their
own guidelines. Film best practice documentation illuminates an interesting fact in analog-todigital preservation: a great deal is known about film and there are consistent guidelines for
understanding decay, film handling, inspection, storage, description, and film-to-film duplication;
while film scanning and digitization is a constantly developing field faced by many of the same
issues as audio and video digitization.
This document is to provide some guidance when preparing to reformat film materials. It is not
intended as a substitute for the advice of experts, but rather as a point of departure for discussions
with vendors.
Section 1: Workflow
Before film material is reformatted or properly stored, important steps must be followed to ensure
the best care and handling and reformatting choices. Films must be identified and a condition
assessment should be completed. Hopefully, identification and assessment are a part of a broad
collection management program but on an item-level, each film is unique and requires varying
levels of attention and preservation considerations. The most important principal when working
with film is "Do No Harm."
Identification
Film gauges:
Most common: Regular 8mm, Super 8, 16mm, 35mm. Less common gauges include: 9.5mm,
28mm
Film stocks:
All film bases have the same basic structure: two layers comprised of a thicker layer of
transparent film base that provides support and a thinner layer that holds the emulsion. Both
layers are vulnerable to decay.
For further information regarding information found below, please refer to NFPF's Film
Preservation Guide.
Film bases:
Cellulose nitrate (35mm), cellulose acetate (35, 38, 16, 9.5, Regular 8 and Super 8mm), and
polyester (35, 16, and some Super 8mm). Once the film base has been determined, serious
consideration must be given to the film's storage and reformatting priority. Cellulose acetate
should be removed from other films due to its vulnerability to vinegar syndrome, requiring A-D
12 August 2008
10
strip testing for acetic acid levels, segregation among varying levels due to the syndrome
spreading to stable acetate films, and prompt cold-storage. Cellulose nitrate must be separated
due to its volatility and also requires cold storage. Polyester is the most stable and will not cause a
threat to other films though proper care and handling is required.
Film elements:
The photographic process of film creation yields various film elements, such as a negative that is
developed, a positive that is projected, A and B rolls for editing, and sometimes reversal film
produced for projection without requiring an internegative.
Black/white and color:
Before the development of color film stock in the 1920s, film prints were hand painted. These
films are still found in collections today and require very specialized care and handling.
Sound tracks and elements:
Sound is part of film in two ways: a track on the film or a separate sound element (film, tape, or
electronic file) that was synced with the film during the editing process. A viewing positive with
a sound track is called a composite print. Composite prints are either optical or magnetic.
Assessment and Storage Preparation
Once the film has been identified, a condition assessment should be completed to determine if the
film has undergone deterioration, the level of decay, and if any other damage has occurred. Film
decay and damage includes: mechanical damage, nitrate decay, acetate decay (vinegar syndrome),
magnetic track deterioration, color fading, shrinkage, and mold, mildew and fungus. Safe film
handling requires specific supplies, some of which Yale already has. For more information, refer
to NFPF's Film Preservation Guide pp. 13-20 and Appendix D.
A-D Strip testing for acetate films
Cellulose acetate film should be separated from any nitrate or polyester film in the collection and
a condition assessment to determine the risk of vinegar syndrome should be carried out for
acetate film. This process is simple, requiring A-D strips produced by the Image Permanence
Institute. For more information, see the Image Permanence Institute Media Storage Quick
Reference and Storage Guide for Acetate Film
Inspecting the film on rewinds
Film rewinds are used for examination of films on reels or cores and for performing a
preservation wind from a reel onto an inert film core. A light box can be placed under the film
passing between the two rewinds, providing staff the opportunity to inspect the film for decay,
damage, and identify elements and previous repairs. Also available are tabletop film viewers.
There are many inspection resources available for use, including standardized inspection work
sheets, footage counters, and repair identifications. Rewinding is also an excellent opportunity to
read the manufacturing codes along the edges of the film (edge codes) as a way to date the film
stock. For more information, refer to NFPF's edge code chart, p. 93.
12 August 2008
11
Any film repair should be done by a trained expert at a qualified lab. This should be done at the
point of being reformatted.
Film cores and cans and labeling
Films are transferred from reels to inert cores and from metal cans to inert cans for long term
storage. All labels identifying information should be kept separately or copied down. See the
NFPF guide for further information on film labeling supplies and best practice. If a film has been
selected for reformatting, it might save time and money to request that the lab rewind the film
onto a core and house in an inert can. Additionally, condition inspection and assessment could be
completed at this stage.
Storage
For excellent guidance and storage specifications for film storage, see the NFPF Preservation
Guide, pp. 59-68 and the IPI Media Storage Quick Reference Guide.
Molecular Sieves
Molecular sieves are used by some archives to absorb the moisture in film cans. There is much
debate regarding their efficacy. I recommend we follow the lead of the Image Permanence
Institute and opt to not use them. If requested, I will provide further information on these and
their use.
Cataloging metadata
See AMIA Compendium of Moving Image Cataloging Practice and Archival Moving Image
Materials: A Cataloging Manual for cataloging guidelines.
Section 2: Technical specifications
Neither the equipment nor the technical expertise required to digitize film is currently at Yale
University. Since there is currently no plan to build a film preservation lab, I will only discuss the
options and considerations in outsourcing film to qualified vendors. Film digitization is an
expensive process that should be done extremely well the first time. Film labs as digitization
vendors are often the leaders in the field and should be used as guiding resources. Currently, the
National Film Preservation Foundation, the only source of broad guidelines for film preservation,
recommends film-to-film preservation transfers and currently requires their grant recipients to
generate two film prints during their preservation project: one film preservation print and one film
access print. Most recipients also request a Digibeta and DVD be made for access copies.
When cared for properly, polyester film is stable and requires less attention than any other format.
However, for ease of use and viewing, other formats are preferable.
Reformatting
Best practices in film digitization are still being developed and will continue to evolve as
technologies change and hard disk storage becomes more affordable. For historically and
culturally valuable titles, film-to-film duplication is still considered the best option, creating a
new viewing print, access copies and master. Film-to-film duplication is expensive and labor
intensive but endures a long life since new film is very stable when properly handled and stored.
12 August 2008
12
For less "valuable" films, high resolution digital file on hard disk storage and Digibeta copies are
recommended for preservation with video and various digital file formats for access copies. I will
not discuss here the preservation issues facing video and hard disk storage but they do inform the
film preservation field's ambivalence towards digitization as preservation.
Preservation formats
DUPLICATING FILM: OPTIONS AND TRADE-OFFS [from NFPF Preservation Guide]
Duplication Approach
Benefits
Limitations
Film, along with
• Long-term protection of
• Most expensive.
film print and
original.
• Film projection and viewing
VHS tape or DVD
Masters can last for years if
equipment required for public
access copy
properly stored. The new
access to film print. Projectors
master is used when new
must be either rented or
copies are required. The
maintained in-house and
access copy shields the
regularly serviced. Flatbed
original, which can be left
editing tables are more
undisturbed in cold and dry
expensive than VHS videotape
storage.
players.
• Print available for screenings
and public service.
• With good-quality
preservation work,
print quality replicates the
sound and visual quality of the
original.
• Film playback equipment
relatively unchanged over
time.
• Proven preservation medium
that has industry standards.
• New print can be copied on
video for access.
VHS tape copy
• Most inexpensive to make.
• Poorer image and sound
only
• Convenient to use.
quality than film.
• Playback equipment
• Inadequate for broadcast
available in most institutions.
or reuse in film production.
• Serves as surrogate for the
• Shorter life span than film.
original, which can be left
• New video copy from
undisturbed in cold and dry
film will have to be made
storage.
within two to three decades.
• Playback equipment likely
to become obsolete.
12 August 2008
13
Betacam SP tape,
along with VHS
tape copy for
routine access
• Serves as surrogate for the
original, which can be left
undisturbed in cold and dry
storage.
• Provides better-quality video
master than VHS for creating
subsequent copies.
• Adequate for broadcast.
Digital Betacam
tape, along with
VHS or DVD copy
for routine
access
• Serves as surrogate for the
original, which can be left
undisturbed in cold and dry
storage.
• Provides better-quality video
master than analog videotape
for creating subsequent copies.
• Digital Betacam tape now in
widespread commercial use.
• Adequate for broadcast.
• Serves as platform for other
digital output media, such as
DVD.
• More expensive than VHS.
• Shorter life span than film.
• New video copy from film
will have to be made within
two to three decades.
• Betacam SP in declining
commercial use.
• Playback equipment likely
to become obsolete.
• More expensive than VHS.
• Shorter life span than film.
• Digital records will need
to be refreshed and
reformatted over time.
• Playback equipment likely
to become obsolete.
Digitization of film has benefits that contribute to the preservation and access of films but best
practices in film digitization are still emerging, and will evolve as technologies change and hard
disk storage becomes more affordable. Digitizing film is a complex undertaking. According to
the University of Maryland's best practice guidelines for digitization projects, these benefits can
contribute to both preservation and access of the films:
Enormous file sizes, time and labor intensive processes, and the instability of the original
object all contribute to the difficulties inherent in digital reformatting of audio and
moving images. There are, however, marked advantages to the digital reformatting of
audio and moving image material that can compensate for the complexity of the process.
These include fragile analog originals receiving less wear and tear due to repeated use,
increased remote access to the content, improved intellectual access through appropriate
metadata creation and increased flexibility for future use.
Currently, when film is digitized, per-frame scans at 4-6k are recommended and Motion JPEG2000 is the optimal file type. For a feature length film at 4k per frame, several terabytes of storage
is required. One second of uncompressed video requires about 21 megabytes of storage space.
Since digitization of films creates digital video files, this portion of the document feeds directly
into the video preservation portion.
12 August 2008
14
Access Options
Local
For local access to digitized film, there two viable options: at the point of a lab transferring film
elements to Digibeta tapes a viewable DVD copy could be produced and/or streaming video
produced for off-site access could be viewed via a digital repository. Optimally, both would be
done for ease of patron requested duplication. If a patron requested a Digibeta copy, one could be
duplicated in-house from the Digibeta access copy. Other access copies can include VHS and
Betacam SP, though Betacam decks are declining in use and VHS has a short life span.
Off-site
For off-site access to digitized film via streaming video online, I recommend we look to
the Folkstreams Guide to Best Practices in Film Digitization prepared in 2007 and funded by the
Institute for Museum and Library Services. Folkstreams is a project of "two goals. One is to build
a national preserve of hard-to-find documentary films about American folk or roots cultures. The
other is to give the films renewed life by digitizing them and making them available on the
project's website, www.folkstreams.net." The guide gives an excellent overview of recommended
formats, sources of recommendations, the web stream production process, and detailed settings
for web stream creation. In summary, they create both MPEG4 and Real streaming files.
Conversion to stream files could occur at the vendor or on campus. Currently, staff expertise and
the necessary hardware and software already exist on campus.
Section 3: Vendor relations
Please refer to Section 3 of the video portion in this document for further information on vendor
relations.
In the past, the library has worked with vendors on film reformatting projects. Vendors used for
previous projects include Vidipax (http://www.vidipax.com/index.php) and others on a perproject basis.
Excellent film preservation labs that are respected in the film preservation field include: Colorlab
(http://www.colorlab.com/), Cineric (http://www.cineric.com/), ScreenSavers
(www.scenesavers.com), Ascent Media (http://www.ascentmedia.com), among others.
Since some labs are not within driving distance, it might be necessary to ship film to laboratories.
Packing and shipping must be well-planned and documented.
For more information about vendor relations, packing and shipping, I recommend consulting the
NFPF Preservation Guide.
Section 4: Examples of successful projects at Yale or other institutions
In 2005, as Archivist at The Josef and Anni Albers Foundation in Bethany, CT, I applied for and
was awarded a National Film Preservation Foundation Basic Preservation Grant for two films
(16mm and 35mm) on Josef Albers. The grant covered laboratory cost for new film preservation
elements and two public access copies, including a new film print and a Digibeta tape. The Albers
12 August 2008
15
Foundation paid for a DVD creation for both films. Other film titles were duplicated with
Foundation funds. Since our film collection was small (4 titles and 65 film cans), I purchased two
upright refrigerators for long term film storage and separated acetate from polyester. Films were
all identified, rewound, rehoused, and cataloged. Film, DVD and Digibeta access copies were
produced.
For case studies, see NFPF's Film Preservation Guide's case studies found throughout and
divided into helpful and convenient sections (duplication, storage, cataloging, legal context,
access, etc.).
Additional resources














Adelstein, Peter Z. IPI Media Storage Quick Reference. Rochester, NY: Image
Permanence Institute, Rochester Institute of Technology, 2004.
AMIA Cataloging and Documentation Committee. AMIA Compendium of Moving Image
Cataloging Practice. Edited by Abigail Leab Martin. Beverly Hills, CA: Association of
Moving Image Archivists; Chicago: Society of American Archivists, 2001.
AMIM Revision Committee and Motion Picture, Broadcasting, and Recorded Sound
Division. Archival Moving Image Materials: A Cataloging Manual. 2nd ed. Washington,
D.C.: Cataloging Distribution Service, Library of Congress, 2000.
Australia National Film and Sound Archive. Film Preservation Handbook
(http://www.nfsa.afc.gov.au/preservation/film_handbook/index.html Film Preservation
Handbook)
Best practice guidelines for digital collections at University of Maryland libraries
(http://www.lib.umd.edu/dcr/publications/best_practice.pdf)
Conservation OnLine Motion Picture Film Preservation
(http://palimpsest.stanford.edu/bytopic/motion-pictures)
Film Forever: The Home Film Preservation Guide (www.filmforever.org)
Folkstreams Guide to Best Practices in Film Digitization, 2007
(http://www.folkstreams.net/bpg/index.html)
Folkstreams’ Video Aids to Film Preservation, 2008 (http://www.folkstreams.net/vafp)
Image Permanence Institute Storage Guide for Acetate Film, 1993.
Independent Media Arts Preservation
(http://www.imappreserve.org/pres_101/index.html)
Moving Image Collections (http://mic.loc.gov)
National Film Preservation Foundation. The Film Preservation Guide: The Basic for
Archives, Libraries, and Museums, 2004
(http://www.filmpreservation.org/preservation/fpg.pdf) [includes excellent glossary of
terms]
Safe Handling, Storage, and Destruction of Nitrate-Based Motion Picture Films. Kodak
Pub. H-182. Rochester, NY: Eastman Kodak Company, 2008.
(http://www.kodak.com/US/en/motion/support/technical/storage_nitrate.jhtml)
12 August 2008
16
Video
Stephen Naron, Archivist, Fortunoff Holocaust Video Archive
This document is divided into four sections, workflow, technical specifications, vendor
information, and examples of successful projects. Much of the information herein is compiled
from other sources, which will be footnoted as appropriate.
Section 1: Workflow best practices
Before reformatting actually takes place, a great deal of collection management must occur. This
should be considered the first step in an extended video reformatting workflow. Obviously, the
curator of the materials being prepared for digitization needs to make some curatorial decisions
about the selection of specific videos for reformation.
Condition assessment
The first step this initial curatorial process is some form of condition assessment. The following
conditions steps are suggested by the Independent Media Arts Preservation website:
“Examine the container. If a box, reel, or cassette is visibly damaged, contents have likely
suffered as well. Stains that suggest liquid contamination may mean accelerated decay or that the
tape is stuck together. Some containers show signs of fungus, which can grow on magnetic tape
in high humidity. This situation can be hazardous and routine inspection should cease until the
scope of a fungal outbreak is understood.
Check for odor. If a tape or film smells musty, it may be contaminated by a fungus. Scents that
can be described as "waxy," "astringent," or "like dirty socks" all indicate the chemical decay of
magnetic tape. "Vinegar syndrome" may be detected by placing small pieces of treated paper
called acid detection strips inside a container. The strips change color to indicate the level of
acidity and thus the severity of degradation. Acid detection strips can detect vinegar syndrome
even before there is a noticeable odor.
Examine the surface and the edges. White powder or crystalline residue on the edge of a tape or
film, or black or brown flakes of oxide inside a container all reveal that degradation has begun. A
gummy residue visible on the surface of a videotape is probably an early sign of binder
breakdown, or "sticky shed syndrome." Damage caused by improper tension during winding
often appears as wavy or scalloped edges, and tape may also be creased or broken due to
mishandling. Dust, dirt, or other foreign deposits may indicate surface contamination, and
discoloration may indicate areas where magnetic pigment material has separated from the
backing.
Identify the format. Polyester film can be distinguished from acetate film by holding the roll up
to a light. Polyester film appears opaque; acetate looks translucent. Polyester will not rip, but
acetate breaks off easily.”
(For further information go to: Information Resources>Internet Resources>Format Guides)
Play the tape. Playback may detect several common problems, including noise, color shift,
distortion, and timing flaws. If a tape does not move through the transport as a result of "sticky
shed" and clogs the heads of the machine, it is deteriorating and must be reformatted. Be careful:
12 August 2008
17
attempts to play back a severely damaged tape before it has been restored places both the tape and
the playback machinery at risk.”1
Video setup
Alignment should be “performed on the system during playback of a tape in order to achieve a
faithful reproduction of the original recording. If this is not done, the signal may be altered and
suffer loss upon playback. If the signal is digitized in this way, the loss is permanent. This is the
reason that bars and tones are placed on many production and broadcast tapes. They offer a
common reference and known value to align with. Many archival tapes do not have bars and
tones. But, even in these cases there are visual cues that should be followed in order to perform
setup and alignment. While this may not be as desirable as bars and tone, it can achieve a much
greater signal to noise ratio and avoid signal distortion. This will add a greater level of integrity.” 2
Metadata capture
Once the materials have been selected for reformatting based on a condition assessment and other
curatorial factors, the next key step is the collection of descriptive, technical, structural and
administrative metadata. Obviously, some of this data is gathered prior and during the condition
assessment, as well as during and after playback. This metadata should include at least the
following elements: 1) original object metadata -- is technical and structural description of the
media. This includes fields such as format, format version, brand, model, track formatting, tape
speed, tape formulation and composition, number of audio channels, and more. 2) digital object
metadata -- is the digital equivalent of original object metadata. Fields include format, version,
bit depth, sample rate, codec, and more. 3) process history metadata -- defines the path between
two iterations of an object, such as the original master and preservation master. This includes
hardware modules, software modules, operators, and actions taken such as cleaning or baking.
This metadata spans the physical and electronic domains. 4) quality metadata -- is information
regarding the quality of the audiovisual signal.3
Quality control
I will leave the technical aspects of digitization for the next section of this paper. An important
part of any best practices, however, must be quality assurance procedures post-digitization. This
is an essential “best practice” for any reformatting process, whether audio-video or paper.
In the case of video, the human eye is the essential tool for quality assurance. Ideally, other
diagnostic equipment can be used to compares various levels (e.g. chroma and luminescence), but
this may not be possible. At the very least, however, a human operator should compare the
original source tape with the digital surrogate. The operator should perform the following checks:
1) bibliographic check -- the content of digital file A is indeed the same as video tape A
2) completeness check – the digital surrogate and the tape should be compared at 3 points,
beginning, middle, and end, to ensure the transfer was complete
1
http://www.imappreserve.org/pres_101/index.html#video
AudioVisual Preservation Solutions. Video Reformatting System Comparative Analysis, March 4th, 2008.
p. 14.
3
AudioVisual Preservation Solutions. Video Reformatting System Comparative Analysis, March 4th, 2008.
p. 7.
2
12 August 2008
18
3) quality check – note any conspicuous differences between audio or video levels
Moreover, once the file has been digitized, an md5 check sum should be used when transferring
digital files, for instance from one server to another.
Section 2: Technical specifications
At this time, it would be very difficult to recommend specific video digitization equipment. The
technology is changing too quickly, and outside of the area of expertise of this paper’s author.
The number of manufacturers of encoding and decoding cards is constantly growing. The table
below focuses on various legacy media and appropriate preservation destination file based
formats for each. There is not a one-size-fits-all solution for video reformatting – there are too
many variables to allow for that. The goal, however, of any digitization project should be to
produce a preservation master file that is the best possible surrogate of the original signal,
whether that original was analog or digital. The detailed technical specifications to meet this goal
for a given source format would require input from a video engineer/video preservation specialist.
We cannot provide a comprehensive list of video formats4, or their ideal digital surrogates. Many
lists of video formats already exist, and some are included in the links section of this paper.
Instead, this paper will focus on the video formats most commonly encountered in the library
system. Those formats include:
Format
Tape
Open reel
1”/2”
Analog or
digital
Analog
Umatic
¾”
Analog
Betacam SP
½”
Analog
VHS
½”
Analog
Hi-8
¼“
Analog
MiniDV
¼”
Digital
DVCam
¼”
Digital
Preservation issues
Requires boutique style
preservation; few working source
decks
Former broadcast standard;
Endangered format; playback
machine population decreasing;
various iterations, such as lo-band,
hi-band, etc.
Former broadcast standard; Larger
machine population, but no longer
used in production
Low quality, non-broadcast
medium;
Field recording medium; delicate,
thin tape;
Uses DV compression, so
maintain same compression
format5; thin tape
Uses DV compression, so
maintain same compression
Preservation
format
Uncompressed
video (MOV)
Uncompressed
video (MOV)
Uncompressed
video (MOV)
Uncompressed
video (MOV)
Uncompressed
video (MOV)
25Mb/s DV file
25Mb/s DV file
4
Choosing digital file formats, as well as maintaining existing files requires analysis of sustainability
factors. See http://www.digitalpreservation.gov/formats/sustain/sustain.shtml www.digitalpreservation.gov
for more on “Sustainability Factors.”
5
Maintaining compression requires stricter ongoing analysis of sustainability factors for that codec. It’s an
added potential barrier that must be monitored to ensure the ability to successfully migrate.
12 August 2008
19
format; thin tape
Digibeta
½”
Digital
Uses compression, but higher
quality than miniDV, still used in
production; Digibeta
decompresses on digital output
and can be captured at full
resolution
Uncompressed
For more information on video formats, see:
http://videopreservation.stanford.edu/dig_mig/video_formats_v4_850.html
Also, care should be taken to maintain the video’s original standard6. It is not recommended to
alter the preservation master file’s standard, for instance from NTSC to PAL. Again, maintaining
an authentic rendering of the original with minimal technical intervention should be at the heart
of any preservation activity.
Preservation file formats
For analog video formats, some form of uncompressed video is recommended.
Another option for preservation reformatting under consideration by the archival community is
Motion JPEG2K. Motion JPEG offers lossless compression of analog video. The file quality is
comparable to the analog original, and the file itself is 2/3 smaller than an uncompressed file. For
instance, 1 hr of video digitized in MJPEG2k requires approximately 33 GB, versus 100 GB per
hour of uncompressed video.
In addition to the file format, some thought should be given to a choice of metadata wrapper for
digital video – if one is required at all. Some institutions choose to forego file wrappers and use
an external database with relevant metadata that refers to the file. File wrappers do add an extra
layer of complexity.
The wrapper formats to be considered for preservation files are QuickTime (.mov) and Material
Exchange Format (MXF). Both formats are platform independent, support many audio and video
codecs, are self documenting – carrying a multitude of useful metadata along with the essence,
have packaging capabilities – support for timecode, subtitles, other “streams” and metadata, and
are flexible in structure and openness, lending to easier extension, portability and migration.7
Access file formats
Again, the selection of access file formats depends to a great extent on the nature of the project,
the audience, and the means of dissemination. The table below lists a few common formats.
6
There are various video standards depending on the geographic location where video was recorded, the
most common are the North American standard (NTSC) and the Phase Alternating Line (PAL), used
mainly in Europe.
7
AudioVisual Preservation Solutions. Video Reformatting System Comparative Analysis, March 4th, 2008.
p. 7.
12 August 2008
20
Format
MPEG2
Compression
Lossy; the basis
for DVD
MPEG4/H.264
Lossy; more
effective
compression
Real player,
windows media
player
DV file
Usually very
lossy
Lossy; basis of
digital video tape
compression
Mbps
Average 2-5; can
support higher
rates
Variable bit rates
Notes
good for desktop uses, and DVD
authoring, but not editing or transfer
of files over a network or the web
supports lower bitrates and is better
for streaming higher quality video
over the net – Apple trailer quality
files
Very low bit
Great for quick streaming low quality
rates
video over the net – YouTube quality
files;
Variable bit rates, Good for editing; very high
usually much
resolution; too big for streaming and
higher
moving across networks effectively
Digital preservation
Of course, once the files have been created, all the relevant best practices of digital preservation
apply. For instance, but not exclusively:
1)
2)
3)
4)
5)
Redundancy: multiple copies of files on different media
Geographic separation: geographically separated
Preservation of metadata within a Digital Repository System
Periodic refreshing of files
Regularly scheduled migration of files to new storage media, and or new digital file
formats as standards evolve
Section 3: Vendor relations
In our experience, there are a few general observations that units should keep in mind when
working with vendors:
1)
2)
3)
4)
Speak to people who have used this vendor for similar projects – get recommendations.
If possible, obtains multiple bids.
Know your specific goals and internal workflow limitations in advance.
And, most importantly, make sure your contract with the vendor is iron clad before you
sign it –describe precisely the work to be done, and include fail-safes, deadlines and
repercussions for not meeting those deadlines.
5) You don’t need to know it all, AV media is complex, but know enough to speak
knowledgably about your materials with the vendor.
In the past, the library has worked with several different vendors on video reformatting projects.
The Video Archive’s experience has primarily been with Vidipax
(http://www.vidipax.com/index.php) and Media Matters LLC (http://www.media-matters.net).
Other preservation labs that seem to follow policies in line with the above best practices for
video, and who can provide uncompressed preservation quality files, are Media Preserve
(http://themediapreserve.com/), Spec Brothers (http://www.specsbros.com/index.html), among
others.
12 August 2008
21
We have also worked closely with Chris Lacinak at AudioVisual Preservation Solution (AVPS;
http://www.avpreserve.com/) – much of this document was drawn from a report Lacinak
compiled a report to help the Fortunoff Video Archive, and other library staff plan for the future
migration of the Fortunoff Video Archive’s collection.
For more information about assessing the preservation characteristics of any lab, and issues
related to outsourcing AV preservation activities, I strongly suggest reading see AVPS’ report
titled Project Outsourcing.8
Section 4: Examples of successful projects at Yale or other institutions
As I mentioned above, there haven’t been any examples within Yale of video preservation that
follow the best practices laid out above. The Fortunoff Video Archive has spent thousands of
dollars reformatting its collection, but until now we have reformatted older tapes to new tapebased formats, as well as some digitization for access only. The next step is to migrate the entire
collection to preservation and access formats, while following a set of best practices similar to the
ones described above.
One project worth noting outside of Yale was Media Matters’ experiment to digitize materials
from the Dance Heritage Coalition’s video collection using MJPEG2K. The report, which covers
many of the essentials in video preservation, as well as real world test results, is available online
as a PDF.9
Additional resources
http://palimpsest.stanford.edu/bytopic/video/
http://www.imappreserve.org/pres_101/index.html
http://videopreservation.stanford.edu/
8
http://www.avpreserve.com/resources/AVPS_Series_Project_Outsourcing.pdf
9
http://www.mediamatters.net/docs/resources/Digital%20Files/MotionJPEG%202000/DigitalVideoPreservation1.pdf
12 August 2008
22
Summary Recommendations
General:
Before beginning a digitization project, media held in long term storage should be evaluated by a
trained professional. Severe damage resulting in the permanent loss of information may occur if
media is played without evaluating its physical condition.
Once media has been digitized, best practices for digital preservation apply. These are typically
understood to be:
1. Redundancy: multiple copies of files on different media
2. Geographic separation: geographically separated
3. Preservation of metadata within a Digital Repository System
4. Periodic refreshing of files
5. Regularly scheduled migration of files to new storage media, and or new digital file
formats as standards evolve
Digitizing Audio:
1. Spoken word recordings should be digitized at a rate of 16 bits, 44.1 kHz.
2. Music and field recordings created to capture a frequency specific sound such as a bird or
animal call, should be digitized at a rate of 24 bits, 96kHz.
3. In either case, a master digital file should be created in a Broadcast Wave Format or .bwf
as a straight copy of the original recording.
4. A duplicate copy master digital file should also be created in a broadcast wave format if
any software tools are to be used to enhance the sound or edit the recording. This digital
copy master is used to create derivative use copies such as an mp3, RealAudio file or to
create copies on compact disc.
Digitizing Film:
For historically and culturally valuable titles, film-to-film duplication is still considered the best
option; creating a new viewing print, access copies and master. Film-to-film duplication is
expensive and labor intensive but endures a long life since new film is very stable when properly
handled and stored. Currently, the National Film Preservation Foundation, the only source of
broad guidelines for film preservation, recommends film-to-film preservation transfers and
currently requires their grant recipients to generate two film prints during their preservation
project: one film preservation print and one film access print.
When film is digitized, per-frame scans at 4-6k are recommended and Motion JPEG-2000 or
.mjp2k, is the optimal file type. For a feature length film at 4k per frame, several terabytes of
storage is required. Digibeta and DVD (Digital Video Disk) formats may also be created as
derivative use copies.
12 August 2008
23
Digitizing Video:
Care should be taken to maintain the video’s original standard10. It is not recommended to alter
the preservation master file’s standard, for instance from NTSC to PAL. Again, maintaining an
authentic rendering of the original with minimal technical intervention should be at the heart of
any preservation activity.
Motion JPEG 2000 is the only file type that offers lossless compression of analog video. The file
quality is comparable to the analog original, and the file itself is 2/3 smaller than an
uncompressed file. For instance, 1 hr of video digitized in motion jpeg 2k requires approximately
33 GB, versus 100 GB per hour of uncompressed video.
The National Audio-Visual Conservation Center in Culpeper, Virginia recommends Motion Jpeg
2000 as the source file digitizing both film and video with an MXF or Material Exchange Format
wrapper.
Wrapper formats to be considered for preservation files are QuickTime (.mov) and Material
Exchange Format (MXF). Both formats are platform independent, support many audio and video
codecs, are self documenting – carrying a multitude of useful metadata along with the essence,
have packaging capabilities – support for timecode, subtitles, other “streams” and metadata, and
are flexible in structure and openness, lending to easier extension, portability and migration.11
10
There are various video standards depending on the geographic location where video was recorded, the
most common are the North American standard (NTSC) and the Phase Alternating Line (PAL), used
mainly in Europe.
11
AudioVisual Preservation Solutions. Video Reformatting System Comparative Analysis, March 4th,
2008. p. 7.
12 August 2008
24
Media Best Practices Task Force Members
Stephen Naron, graduated with a Master of Science in Information Studies from the School of
Information at the University of Texas at Austin in 2003. He has worked at the Fortunoff Video
Archive for Holocaust Testimonies at Yale University for more than 4 years. A member of AMIA
and SAA for many years, Stephen serves on AMIA’s preservation committee; has attended many
workshops and conferences related to the preservation of audiovisual cultural heritage, such as
the Joint Technical Symposium; assisted with the planning and implementation of the beta test of
Media Matter’s system for the automated migration of media assets (SAMMA) at Yale; and is a
member of the library team that is planning the digitization of the Fortunoff Video Archive’s
entire holdings, which comprises more than 12,000 hours of videotape, into preservation and
access formats.
David Walls has an MLIS, and a Certificate of Advanced Study in the Preservation
Administration of Library and Archival Materials from the University of Texas at Austin. He has
been a Preservation Librarian at Yale since 1996. In 2006, the Core Reformatting Program
within the Preservation Department was reorganized as Reformatting and Media Preservation, to
meet the increasing demand for media preservation and reformatting. Since that time he has
worked with curators on a wide variety of reformatting projects including the Cuban Film Project
and the Islamic Tape Project. He serves on the Strategic Planning Committee of the Northeast
Document Conservation Center and is a member of the Society for Imaging Science and
Technology, the Association of Moving Image Archivists, and the Association for Recorded
Sound Collections.
Molly Wheeler graduated with a Master of Science in Information Studies with a Certificate of
Advanced Study in Preservation Administration of Library and Archival Materials from the
School of Information at the University of Texas at Austin in 2003. She has worked at the
Beinecke Rare Book and Manuscript Library at Yale University for 14 months and before joining
Yale worked for 3.5 years as archivist at The Josef and Anni Albers Foundation in Bethany, CT.
While a student at UT, Molly worked as an intern at the Harry Ransom Humanities Research
Center, where she built an audio reformatting lab and helped design and implement an audio
preservation program. A member of the Association of Moving Image Archivists for many years,
Molly serves on AMIA’s Strategic Plan Implementation Task Force and the Small Gauge and
Amateur Film Interest Group. Molly is on the Board of Directors for the Center for Home
Movies, a registered 501(c)(3) public benefit corporation, and organizes the Connecticut Home
Movie Day. While archivist at the Albers Foundation, she received a 2005 National Film
Preservation Foundation preservation grant and as a volunteer she recently received a 2008 NFPF
preservation grant on behalf of the Yale Film Study Center for a film by Mary Ellen Bute. A
member of the Society of American Archivists, she has chaired and participated in sessions and
various sections and is currently the chair of the New England Archivists Outreach Committee.
12 August 2008
25