Visualizing Social Photography
Lev Manovich
Forthcoming in Aperture magazine #214 (2014).
Last summer the Museum of Modern Art in New York asked the Software Studies Initiative,
a program I started in 2007 to explore how visualization could be used as a research tool
and perhaps a means to present their photography collection in a novel way. We received
access to approximately twenty thousand digitized photographs, which we then combined
using our software in a single very high resolution image. This allowed us to view all the
images at once, scrolling from those dating from the dawn of the medium to the present,
spanning in the meantime countries, genres, techniques, and photographers’ diverse
sensibilities. Practically every iconic photograph was included—images I had seen
reproduced repeatedly. My ability to easily zoom in on each image and study its details, or
zoom out to see it in its totality, was almost a religious experience.
Looking at twenty thousand photographs simultaneously may sound amazing, since
even the largest museum gallery includes about a hundred works at the most. And yet,
MoMA’s collection, by twentieth-century standards, is meager compared with the massive
reservoirs of photographs available on media sharing sites such as Instagram, Flickr, and
500px. (Instagram alone already contains over one billion photographs, while Facebook
users upload over ten billion images every month.) The rise of “social photography,”
pioneered by Flickr in 2005, has opened fascinating new possibilities for cultural research.
The photo-universe created by hundreds of millions of people might be considered a megadocumentary, without a script or director, but this documentary’s scale requires
computational tools—databases, search engines, visualization—in order to “watch.”
Mining the constituent parts of this “documentary” can teach us about vernacular
photography and habits that govern digital image making. When people photograph one
another, do they privilege particular framing styles, ala a professional photographer? Do
tourists visiting New York photograph the same subjects; are their choices culturally
determined? And when they do photograph the same subject (for example, plants on the
High Line Park in West Manhattan), do they use the same techniques?
To begin answering these questions, we can use computers to analyze the visual
attributes and content of millions of photographs and their accompanying descriptions,
tags, geographical coordinates, upload dates and times, and then interpret the results.
While this research only began a few years ago, there are already a number of interesting
projects that point toward future “computational visual sociology” and “computational
photo criticism.” In 2009, David Crandall and his colleagues from the Computer Science
Department at Cornell University published a paper titled Mapping the World’s Photos
based on analysis of approximately thirty-five million Flickr photographs. As part of their
research, they created a map consisting of locations where all these images were taken
(world heat map). Areas with more photos appear brighter, while those with fewer
photographs are dark. Not surprisingly, the United States and Western Europe are brightly
illuminated, while the rest of the world remains in the dark, indicating more sporadic
coverage. But the map also reveals some unexpected patterns—the shorelines of most
continents are very bright, while the interiors of the continents, with the notable
exceptions of the States and Western Europe, remain completely dark.
Using their collected photo set, Crandall and his team also determined the most
photographed locations in twenty-five metropolitan areas. This led to novel discoveries—
New York’s fifth most photographed location was the midtown Apple store; Tate Modern
ranked number two in London. A photo-mapping project Locals and Tourists created in
2010 by data artist and software developer Eric Fisher addressed a question likely
prompted by such information: how many of these images were captured by tourists or
local residents, and how does this distinction can reveal different patterns? Fisher’s “Locals
and Tourists” plotted the locations of large numbers of Flickr photographs by using color to
indicate who took them: blue pictures by locals, red pictures by tourists, yellow pictures
might have been made by either group. In total he mapped 136 cities, then shared these
maps on Flickr. In his map of London we see how tourists frequent a few well-known sites,
all in central London, while locals cover the whole city, but document less assiduously.
These pioneering projects use metadata to reveal telling patterns in social
photography. However, they did not use actual images in their visualizations, a
practice first explored, to my knowledge, by artist James Salavon. For projects such
as Every Playboy Centerfold, 1988–1997 and Homes for Sale, 1999-2002, Salavon
composited a number of images to reveal the photographic conventions used to
represent particular subjects. His more recent work, Good and Evil '12, 2012,
consists of two panels, each showing approximately twenty-five thousand
photographs returned by Bing image search for the one-hundred most positive or
negative words in English.
Media artists like Salavon demonstrate how visualization may uncover
patterns in the content of large image collections. In 2007, I set up a research lab to
explore this idea further and to develop open-source visualization tools that can be
used by anyone working with images—art historians, film and media scholars,
curators. One of our software tools can analyze visual properties (such as contrast,
gray scale, texture, dominant colors, line orientations) and some dimensions of
content (presence and positions of faces and bodies) of any number of images.
Another tool can use results of this analysis to position all images in a single highresolution visualization sorted by their properties and metadata. We used these
tools to visualize a variety of image collections, ranging from every cover of Time
magazine between 1923 and 2009, a total of 4,535 covers, to one-million manga
pages.
For our recent project, Phototrails, I’m working with Art History PhD student
Nadav Hochman and designer/programmer Jay Chow to explore patterns among
millions of photos uploaded to social media sites. We downloaded and analyzed 2.3
million Instagram photographs from thirteen global cities. One of our visualizations
shows 53,498 photos shared by people on Instagram in Tokyo over a few
consecutive days. The progression of people’s dominant activities throughout the
day—working, having dinner, going out—is reflected in changing colors and relative
brightness of images. No day is the same. Some are shorter than others, or the
progression between different activities is very gradual, while in others it is sharper.
Together, these photos create an “aggregate documentary” of Tokyo—a portrait of
the city’s changing temporal patterns aggregated from thousands of documented
activities.
But are aggregated documentaries new? Dziga Vertov’s 1929 experimental
film Man with a Movie Camera, which portrays a single day in the life of a Soviet city,
might be considered a precursor to the form. The film combines footage shot in
three separate Ukrainian cities—Odessa, Khartiv, and Kiev—over a three-year
period. Vertov wanted to communicate particular ideas such as construction of a
communist society that guided the selection and editing of his footage. Our
visualizations of human habits rendered through Instagram photographs do not
reflect a single directorial point of a view. Even so, they are as subjective as more
traditional photography. Just as a photographer decides on framing and perspective,
we make formal decisions about how to map the images, organizing them by upload
dates, or average color, or brightness, and so on. But by visualizing the same set of
images in multiple ways, we remind viewers than no single visualization offers an
objective interpretation, just as no single, traditional documentary image could ever
be considered neutral. Instead, the diversity of the Instagram photographs
highlights the variety of complex patterns of life unfolding in cities that can never be
fully visually captured in a single visualization, despite our ability use millions of
Instagram photographs.
( November 2013.)