>> Andy Wilson: So my name is Andy Wilson. I'm manager of the Natural Interaction Research
Group, and it was my pleasure to have James here for the last three months. And so he visited
here, I guess last year, and I was just struck by how much stuff we had in common in terms of
the way we thought about computation and sensing and, in particular, some of the really great
things he was doing with depth cameras. For many people, including people in the company
here, it's all about skeletal tracking and tracking points and hand tracking; and his pursuit is
completely different and much more nuanced and very interesting to me. And so he's going to
talk a bit about that work. So a little bit about him, he is from New York, but he's a man of the
world. He’d done a lot of different things all over the place and exhibited all over the world. He
has a lecturer position at ITP, NYU’s ITP, and has been working with, what’s the group at NC
New? Creative>> James George: The Studio for Creative Inquiry.
>> Andy Wilson: Creative inquiry, and has been involved with some of other groups and is
particularly active in the Open Frameworks community as well. So anyways, so I think this will
be a really nice time to hear about things other than the things he showed yesterday including
the RGBD Toolkit and Clouds Documentary and all sorts of other cool stuff. Great. Thanks.
>> James George: Thanks for the intro. It’s been wonderful spending the last three months
with everyone here; and I've kind of developed a lot of new ideas thinking about photography
in creative practice and coding, while I was here so I'm trying out a new talk. I will talk a little
bit about my work on display, but I'll save it till the end and probably just kind of rush through
it. So I have about an hour, right? So since it's a new talk I might be a little wonky on time. But
I really want to talk about what I see something that's happening with image making right now
in culture and talk a little bit with how I'm engaging with it and how other people are engaging
with it.
So I think this is a really cool exhibition or project to sort of frame this discussion. So this is a
project by Erik Kessels, it was done in the 2011; it’s called 24 HRS. It's a room full of photo print
of every photo that was uploaded to Flickr within a 24-hour period. And as you can see it
actually fills a very, very vast space, something that you can't even see the ends of it and it’s full
of random curiosities. And it goes to show how Erik Kessels, he’s a photographer, he's trained
in photography and he’s starting to sort of deal with or grapple with the fact of how much
imagery our culture is producing and uploading at every moment. And there's this statistic that
I heard that I like to talk about with this is that 10 percent of all photographs were taken this
year. And photography is not super new in terms of the big bang theory and all these large
history, but last year is only a fraction of a percent of photography’s history, and yet it occupies
a large percent of what the photographs have been taken.
So another project, another appropriation project is Sunsets by Penelope Umbrico, and she's
been obsessively collecting pictures of sunsets from Flickr, and she shows them as prints. And I
think this is a natural reaction to this large amount of imagery that is in, that we're coming is to
collect it and organize it and categorize it; but there's something else that's happening which is
about synthesis and that's something that's happening here a lot. So I'll get into that. But I
want to frame this discussion with a story by Borges, which is always great. And this is a very
short story. The entirety of the story is this paragraph. The translation is closely translated to
on exactitude in science. And I'll just read it aloud.
So he starts, “In that Empire, the Art of Cartography attained such Perfection that the map of a
single Province occupied the entirety of a City, and the map of the Empire the entirety of a
Province. In time, those Unconscionable Maps no longer satisfied and the Cartographers Guilds
struck a Map of the Empire whose sizes that of the Empire and which coincided point for point
with it. Now the following Generations, who were not so fond of the Study of Cartography as
their Forebears had been, saw that vast Map as Useless and not without some Pitilessness was
it that they delivered it up to the Inclemencies of Sun and Winters. In the Deserts of the West,
still today, there are Tattered Ruins of that Map, inhabited by Animals and Beggars; in all the
land there is no other Relic of the Disciplines of Geography.”
Think what will happen to our work as we go forward with our obsessions. So I'm titling the
beginning of this as the idea of computational photography within artistic practice. So a lot of
people here work with the computer science discipline of computational photography, but I
want to show specific projects that you may not have encountered that are how artists or
people, photographers are working with computation to synthesize different ways of looking at
the world and in a lot of ways grappling with new forms of image making and the glut of
imagery and available technology.
So I'm going to start this talk, I'm actually talking as less of an artist and more of a curator and
show several projects that are not my own but that I relate to and think are quite profound. I
split it into two sections. The first is looking outward, and so this is a way of doing panoramic
photos that create these things called tiny worlds. Normally we're used to seeing panoramas
kind of all around us, but it’s a certain projection. You can do a spherical projection and you
get, so this is a panorama of New York City from one perspective, but it makes you feel as if the
whole world is contained within a small planet. So I'll be jumping between the website and my
slides really quickly. So it will be quite fluid.
So the first project that I love so much in the looking outward series is a project in 2011 by Jon
Rafman. Jon Rafman’s a media artist and sort of net artist and it’s called 9-Eyes. And it was
made in reaction to, when Street View was first, oh is this link not going to link me? Okay. So
I'll link to this project. Oh, man. So this is a street photography project really, but also an image
appropriation project because Jon Rafman as a photographer didn't take any of these images,
but instead he spent excessive amounts of hours browsing through Google, oh you can't see it,
browsing through Google Street View to find the most strange and odd and magical moments
that the Street View car captured. And there's thousands of them. Well, maybe hundreds. So
car accidents that may or may not have been caused by the presence of the Street View car,
really beautiful moments like this Saran wrap being lead behind this car in some tropical
setting, more car accidents, people acting strangely, covering their faces, reacting to the car
itself, and in a lot of ways it draws from traditions of the street photography but he never really
took these photos, who took the photos here? Was it Google, is it Jon Rafman, or is it some
sort of ad hoc collaboration?
There’s one of my favorites on here which is way down, is a photograph of an elephant on a
safari plane looking at the car and its face has been blurred out by face detection software. So
we don't have any idea of the identity of that elephant. So these are fun. So the next project is
called Google Faces by Cedric Keiffer, and he is a Berlin-based artist running a studio called
onformative; and he labeled this project, an independent searching agent hovering the world to
do spot all faces that are hidden on earth. And it draws a little bit on this idea of pareidolia
which is the human concept and phenomenon of seeing faces and things where they aren’t
actually faces, so the fact that you can draw two circles and a sort of upturned line and
everybody perceives it as a smiley face is that phenomenon. And now that we have advanced
face detection software, I'm losing my cursor, computers have started to experience the joy of
this phenomenon as well.
So what Cedric did was he wrote an algorithm to download Google Earth tiles and just look. Is
there a face? Do you see a face? Do you see a face? Every once in a while he would find a
face. Or rather, the search algorithm would find a face. And inevitably when you looked at
those tiles that it would come up with you too would see something a bit like a face. And so he
has this running, constantly searching for sort of the human, the man in the moon it could be
said, or perhaps the man on earth now.
I'm going quickly because there's a lot of these. And maybe not so intelligent as or as crafty as
Cedric's but still is equally profound are Postcards from Google Earth which is the 2010 project
by Clement Valla. And he, in a similar way to Jon Rafman, searched Google Earth looking for
strange phenomenons that revealed the structure of how the algorithm worked and how the
texture mapping integrated with the way you’d perceive these images, so most interestingly
the way that bridges were mapped as they went over vast terrains because Google has a mesh.
It knows the shape of the earth, but it doesn't know subtleties like a man-made bridge that
would go across the canal. So when you get, so he created an archive of these phenomenon
and he titled them Postcards; and there's actually a tour that you can take on Google Earth
which zooms into each one of them. And recently, actually as a side note, a great wind for
digital arts in the art world, these Postcards were sold for quite a great sum as the first digital
art auction. So that was fun and a good move for Clement. So these bridges get mapped into
the canals, and I think it's unsettling because of the way it makes us feel as we see these
bridges sort of crumbling under the weight of our own systems.
This one's pretty fun. So Teehan-Lax, they use Open Frameworks, they’re an experimental lab;
they do advertising and art projects in New York, and they made a project called Street View
Hyperlapse which let you take a digital tour along long stretch of highway's inside, along Google
Street View. I really wish my links worked out of the thing. Sorry, guys. So I'll show just a, you
can create your own with these, you can choose two points on a highway and it will generate a
video for you, but they also have a nice little bumper video for it with some road trip music to
boot.
[demo]
So when I start to see this I start to wonder if that map is starting to become point for point.
[demo]
>>: [inaudible]?
>> James George: Yeah. I think you go in and you design the camera moves that are sematic to
follow certain things and so, yeah. There's some film making at play. So you can watch a lot of
these. You can generate your own.
>>: [inaudible]?
>> James George: Yeah. Definitely. But that comes a lot from also the shape of the road. So a
final note, which I think is really beautiful and starts to blend into the next subject is this project
that just came out from a ScanLab Projects called Frozen Relics. ScanLab is an interesting
group. They're based in the UK, and they, at first I was skeptical and then I thought they were
just bros with a Lidar scanner; but they are studying in architecture, and they are able to use
this 3-D scanner to engage with a lot of contemporary issues. I think this is the most profound
of their projects to date.
So it's called Frozen Relics, and they’re working with environmentalists to take their scanner to
the Arctic and take a snapshot of the state of the ice; and with these high-definition scans,
millimeter accurate scans of miles and miles of ice they re-created, give myself some more
room, so this is an example of what these scans look like. They re-created the current state of
the ice and were able to fabricate it in a gallery actually out of ice. So you could see first-hand,
of course it's very sort of didactic, but it’s there dripping and melting. But it's an accurate
portrait of the current state of the environment, and is both a scary but also laden with hope
because with these technologies perhaps we are able to intervene and actually know the
damage that we are doing. So it's sort of an open question. So these are actual to scale models
made in ice in a gallery that you can see, and each one of these points is sort of the scanner’s,
where the scanner was, so it’s an artifact of the capture process. It can’t see right underneath
it.
So that bleeds a little bit into, through the technology into the next theme which is looking
inward. So we're not just using photography to look at our environment, but we also use it to
look at ourselves in portraiture, something I'm really interested in. And there’s artists working
in this domain that are blending traditional sculpture and portraiture with a new media. And so
this is one of the best examples, I think, this is called Triple Portrait of E which is a set of 3-D
laser scans, several of them combined into one of mesh and 3-D printed. But the delicacy with
which Sophie approaches her materials gives it the quality of sculpture, and it feels as if it were
a cast of a face. And I'll show you just a few more of her projects. So all of these are collections
of several scans that are 3-D printed and assembled into these portraits and sometimes cast
out of different materials.
>>: [inaudible] real person?
>> James George: Yes. They’re scans from a real person. It’s 3-D scans. It uses a laser scanner.
It's much like a Connect. Another group that is influential to me personally is the OpenEnded
Group, which are three collaborators out of MIT now working in New York, I believe, who are
really interested at the intersection of sensing in cinema. They created this work called
Upending which blends several different capture techniques into creating these very distilled
domestic moments. So I'll show just a quick clip of that. I'm at noon? I'm done at noon?
>>: We actually have [inaudible].
>> James George: I might take it, it seems like. And I just want to point out that all of these
projects have been done in the last three years. Let’s see if there's an excerpt here. So this is
pre, sort of depth-sensing, using feature tracking in a different ways of deriving 3-D data out of
2-D reconstruction. But the example of synthesis, of many images together to create
something, that's more than their combination. You might want to get the lights just a little bit
for the projection. Thank you. I think there's music as well.
[demo with music]
>> James George: So as artists start to work in these domains I notice often that the languages
of animation, in 3-D as well as hand animation and photography start to blend quite deeply and
become indistinguishable; and I think that's a trend that we are seeing in a lot of these fields is
that synthesis and creating something from nothing and sampling things from the world and
creating meaning out of that are, it’s becoming quite blurry which is fun because languages are
starting to really talk to each other. If you do computer graphics or if you're a photographer,
now there's a lot of place for discussion where previously there might not have been.
>>: [inaudible]?
>> James George: Say again?
>>: Do you see any link between these and data visualization as on the web?
>> James George: Yeah. I like to describe this type of work as a data visualization where your
data set is time, color, and space. So you're really, it is a visualization of data, but the meaning
is not necessarily on semantics but it's on a perception of the image that it creates. But it is
data visualization. I mean even taking a two-dimensional photograph and showing it on your
computer is a visualization of data. But it is a semantic distinction. I think the subject matter
here is what makes it interesting the way we are using it as a lens. So I'll give that a pause.
And this isn’t just, the thing that I think distinguishes this and makes it significant too is it's not
just sort of artists working in rarified ways or with really highly specialized tools, these are
becoming quite accessible. And there's a lot of software that's being released for free or for
cheap that lets you just as a tinkerer or hobbyist experiment with these types of media as much
as like hobbyists and photographers would be taking snapshots. And I think the most, the
coolest example of this is the 123D Catch Program by Autodesk.
So 123D Catch, you can see here it’s a picture of a child and he’s holding it 3-D printed portrait
of himself. It's a pretty uncanny likeness, and it really touches on going back to the first image
of the room of photographs, well, yeah, so you can take a room of photographs and you can
print them all out on snapshots and say well this is what's happening, or you can look at
something like this and say well this actually is also a crazy collection of photographs. But you
never look at the image as a 2-D picture. In fact, you run it through a system and out of it you
get a three-dimensional model of the world. So you combine several photographs along with
knowledge about how those photographs were taken and something new and different comes
out; and the snapshot as a unit starts to become irrelevant as a photograph, as an individual
thing. But really the understanding that you get from the collection of them starts become
what's important. And it's not, it's also fun and accessible like there's people just want to do
this. People want to see themselves this way and they want to experiment with it. So this is
hobbyist culture of 123D Catch is pretty amazing. You can go on there and see a lot of things
that people are making.
And so actually at the bottom, it's kind of hard to see here, but you can see all the individual
pictures that were taken and this has become like a fully 3-D model that you can rotate like an
action figure basically. And this is being taken to the extreme by some people. And it kind of,
this is where it becomes a little science-fiction. My favorite example of this is this guy, Lee
Perry Smith, and I've been following his work for the past two years, and he has a company
called Infinite Realities, and he's been publishing a lot of very provocative and silly and fun but
incredibly accurate 3-D, full 3-D body scans using combinations of photographs. And recently
he’s been publishing a lot of his stuff in 3-Ds in the Oculus Rift[phonetic], so you can watch it in
an immersive thing, I want to show you one of his videos. So these are all photographs
ostensibly of human beings taken, but synthesized 3-D models and textured.
[demo with music]
>> James George: So the reason there's two screens here is because this is made for a virtual
reality headset so you'd actually see it as if the whole world was around you. And the
movement of the head is actually a person's natural movement of their head with the device
that he's immersed in and responding to so that way you can see it reveals the material a bit,
that it’s actually just a 3-D model. So it starts to suggest the few science-fiction notions like
teleportation, maybe some things, feelings of the matrix are the things that have been
suggested maybe starting 10 years ago, Neromancer[phonetic], William Gibson and the
Cyberpunks.
Also, on that tip Lee has released the web store called triplegangers.com, where you can
actually purchase these people and do whatever you want with them. So a full rendered, I
don't think they are rigged to be animated, but they might be, but 3-D people for sale. So
whatever fantasies that suggests.
>>: I mean you get like figurines [inaudible]?
>> James George: Well, you can 3-D print them if you wanted, but you're getting the data.
>>: Okay.
>> James George: Yeah. So now we are in definite science-fiction domain so we're going to go
all the way, and I'm going to start this off with talking one of my heroes and people that I love
to follow and snark with on Twitter is Bruce Sterling. Bruce Sterling is part of the Cyberpunk
Movement. He's somewhat eclipsed by William Gibson. They were buddies; they were writing
future, the future of technology in the 80s, the 70s and 80s. And right now Bruce is actually
sort of retired from science-fiction and he started, he runs a blog on the wire called Beyond the
Beyond. He follows tech art. He also started calling himself a design fiction author where he
actually talks about the immediate future of technology, and these are a few of his books which
I recommend you read. Shaping Things is very influential on me. And the first time I
encountered Bruce it was actually in person and he was giving a keynote speech at the Vimeo
Awards Festival in 2010, kind of when a lot of this was kicking off, right? And he was talking
about the camera of the future. So he was talking about video and where it was going and he
went on this rant to describe what he believes cameras in the future will look like. I'm going to
play for you an excerpt from this talk where he makes that, where he describes that camera.
[demo]
“They would look something like this microphone except probably a lot smaller. And this thing
here at the head of the microphone would be the lens, except that it's not a lens, it's just a
piece of a black fabric or glass that's an absorptive surface that simply absorbs every photon
that touches it from any angle. And then in order to take a picture I simply tell the system to
calculate what that picture would have looked like from that angle at that moment. You see? I
just send it as a computational problem out into the cloud wirelessly.”
>> James George: So Bruce is describing is something that is taking this glut of imagery to an
extreme, and really he's referencing in a way Borge’s story of the point from point map of the
empire, but instead of it being a map it's actually a database of all time, color, and space that's
ever been experienced. And if you want to take a picture you model a lens in your computer
software, you put it somewhere in space and you give it a timestamp, and then you get a
photograph of that moment. Obviously it's an extreme version, but we are starting to see
inklings of that notion coming up in reality. And actually, probably two weeks after Bruce gave
this talk, this device was released, and when it was released it was quickly reversed in the code,
the way that it talked to a computer was quickly reversed engineered or I should say it’s the
Microsoft Connect, most people are here are familiar with it, but it’s a videogame controller
that relies on gestural input to be able to interact with computers. So it actually perceives the
way people are standing and as Andy mentioned, it’s very focused on skeleton tracking on
looking at humans and knowing how they stand, but what it did also was it took this really
rarefied technology of 3-D sensing and put it into people's homes and made a common idiom
that everybody had a 3-D scanner in their house now, and that changed the way that we think
about these things especially because it became easy to reverse engineer, plug into your
computer, and do what you want with it.
Now when this came out Open Framework Community, which was a part of it, which was a
coding community that work in Open Source, we created a way to interface with the device and
started experimenting with it. And this was one of the first images that was published online in
2010 after the device was released. And when I saw this, this is a photograph of my friend Kyle
McDonald, and he's looking into the lens of the camera, but actually camera that we're seeing,
the perspective we are seeing is off to the side. And just, it looks very frail and new and there's
a lot of missing pieces, it's only black and white, it's very [inaudible], but the suggestion that
you can look at a picture from a different angle than when it was taken a seemed to resonate
with me from what I had just heard Bruce Sterling talking about and so I wanted to work in this
domain. I wanted to do something with it. So I'm blending now into my own work and in
reference to the other stuff.
So I had just moved to New York at the time, I had been living there for about a year; and I was
spending a lot of time in the subway and thinking about urban spaces and cities and commonly
sort of brushing shoulders through places like this, and I had just heard a story on NPR about a
surveillance system that had been commissioned from Lockheed Martin by the MTA, and this
surveillance system was unprecedented in scale. And what made it really unique was it was the
first security surveillance system where no people would be looking at the images from the
camera. Instead, all of these cameras that were being installed were getting piped directly into
computers running software that with analyzing images and trying to synthesize whether there
was a risk at hand; and they spent millions of dollars installing this hardware and developing the
software, and when they deployed it, it failed to function at all and it resulted in huge lawsuits
on both sides. And this is actually, this is one of the only legal documents I've enjoyed reading
which is the court hearing from this lawsuit, and it reads like a William Gibson novel. It's like
talking about the failure of artificial intelligence is to keep the world safe, and so it’s quite a
funny story. And it starts to suggest where are the actual innovations coming from? And is it
going to go the direction that it’s been anticipated, the failure of artificial intelligence to follow
through?
So I took this as an inspiration with the Connect, and along with the photographer friend of
mine, Alexander Porter, we took the device and we strapped a camera to it so that we could
take photographs as well and along with a car battery in my laptop in my backpack, we went to
the subway and started taking photos to sort of imagine what the system would have looked
like had it succeeded or maybe how it would've seen the world. And when we started pulling
these images back onto our computer and rendering with software I was finding myself working
in this domain which is this combining language of computer graphics, polygons, lines, triangles,
texture mapping, but all of it was being done using stuff that was taken from the world. So I
was able to mix these two worlds and create this portraiture that felt very profound for our
current era, us being composed of digital system and seen through robot eyes, really.
And again, it started to suggest this camera of the future where I can take this, maybe we can
lower the lights for this one because the imagery is fairly dark? Thank you. This photograph is
actually the same moment in time and is the same photograph as this, but they are shown from
two different angles and actually decisions can be made after the fact that entirely alter what
parts of the image are there and what parts are missing. And so we were interested in sharing
this notion with people and we encountered a problem when we were combining these two
devices which was that things didn't really line up and there was an engineering problem at
hand which is how do you combine these cameras together? How do you start actually
synthesizing more and exploring this medium to a greater degree?
And, just briefly, the engineering problem here was that if you have two lenses that are from
two different perspectives it's actually a three-dimensional part problem to get the color from
the video camera, the photograph to line up with the depth, the 3-D information from the
Connect. So this was something that we needed to solve. And in my community the way that
we solve these problems is we meet together at conferences or hack-a-thons or kind of ad hoc
locations and we sort of distribute it all over the world to people I collaborate with and we kind
of hack on stuff together. And this time, Golon Levin,[phonetic] who is a big part of why I'm
here, had created a conference called Art and Code 3-D; and this was in 2011, and this were a
bunch of people who had been hacking on 3-D sensing got together and started working on
projects.
And I like to equate these types of meetings to something more like this which is people
gathering around a campfire; and Open Frameworks, the Open Source community, we are
always collaborating online. So the Internet is kind of like our campfire where we are sharing
stories and swapping ideas, but it feels very communal. It feels very almost tribal. You look out
for each other, you get each other jobs; you help keep your independence, really. And you're
open with your ideas. And so at this conference I work with several people, and we kind of
cracked the code on how to take this scanner camera combination that I had taken these
subway photographs with it actually create video out of it. And we were like well, what are we
going to use this for? And as the natural fit we decided to interview the people that were at the
conference who were experimenting with art and technology and really create, use this form of
portraiture image making to tell the story of the hacker ethos and artists that are using
technology to engage with new ideas. And so I'll show you an excerpt from the first video we
produced using this technique. And it's interviews with people working in this space.
[demo with music and interviews]
>>: And he’s basically explaining how to database works. And he says there’s point here and
you go around and there's another [inaudible] center there and another one there, and
because this is closer than this is one which is matched. He explains everything like that.
>>: Is there an experience, a visual experience that you desire to have?
>>: Yes, data transportation. Yeah. That's, there are many things. There is so much I shouldn't
talk about. But data transportation because there was a time when they say, why do you do
videoconferences? You all need to meet. But they don't work so well. You still need to be
with the people to really discuss and work on project. You can do part of it by
videoconferences, but it has its limits. So I have to travel, and I'm sick of traveling, and
[inaudible] in the world and we travel trains and [inaudible] transportation. Is it possible?
Going to be possible? I really want it.
>>: You could start off with the point [inaudible] of this and, I don't know, at some point when I
say talking about what are our physical systems ultimate point? You could just flush it or turn
into water or something like that just to illustrate the thing you've been talking about.
>> James George: So that's an excerpt, and since doing that sort of preview experiment we've
expanded the interviews, and we're working on a feature-length, you have a question?
>>: Yes, I have a question. So where does all this music come from? Is there like a beat sample
of like?
>> James George: That track?
>>: I mean all these, there's [inaudible] right?
>> James George: Right.
>>: There’s sort of like a future sound, like how do you decide what that music is? Where does
it come from?
>> James George: I don’t know. It's all sort of in the aesthetic, right? And it's weird. Actually,
I'll get to it in just a little bit, but it's based on the similar responses to technological movements
and that's my perception. Like synthesis is almost like the celebration of technological aesthetic
which is dangerous but often what underlies a lot of this stuff. I mean that track was like an
amalgamation of [inaudible] and this guy, Scott Gibbons. We mix it a bunch, but yeah.
>>: Is there an experiment to connected to the Internet to like get that type of sound? Like
office or something like that?
>> James George: Oh. I don't know. Maybe Ambient Electronic? Or Vortek. Daft Punk?
>>: There you go.
>> James George: So, yeah. Good question. I'm very visual; audio is often not something I
notice, but it's important. So just briefly, Clouds, which is actually set to premiere very soon,
we've been working on it in the nighttime while I've been here, in its final form it's an
interactive documentary where you can browse through this archive of 45 interviews taken
much like the way you just saw but you're allowed to follow your own interests through asking
questions to the system and it will generate for you a sequence of responses that are based in
3-D and have interactive systems with them. So it will be quite fun. We are actually targeting
the Oculus Rift for that as well. So I’m as guilty as the rest of them.
But feeding into the story more of this talk, in addition to creating the film, and as we’ve been
developing the film we've been releasing all of our experiments Open Source in the form of this
thing which is called the RGBD Toolkit which is a name that I'm sorry to be stuck with, we are
changing it very soon; so it’s an Open Source software kind of like Final Cut but freeware,
shareware program you can download and if you have a camera and a Connect and you don't
need to know code you can start to experiment with this form of image making and make your
own sort of 3-D volumetric wire frame movies. And this is something that has been quite fun to
track the community, and we've actually introduced me to a lot of people who are interested in
this world as well and who are using it. So we collect a lot of these projects online, we have a
website, people are using it for all sorts of different things, and there's three projects that I
want to show, just have like, I have little cuts of them which kind of lay out the terrain of what
people are using it for and the variety.
So the first that I like the most is this family portrait by Trevor Ames, and it's his first
experiment, and he sat down with his mom and dad on the couch and they did a family portrait
which is an app. This couch and space is pre-apped for 2012. This is a prayer done by this
World Vision International, which is a very theist outreach program based out of I think
Australia. So obviously they interpreted this aesthetic as being reverent and transcendent. And
this is a porn film made by a Mexican artist with his girlfriend which you can watch online. So
it's sort of ranging from the pious to the profane to the benal. It's being used everywhere in
between. And actually we had a really great moment last week where I was surprised to wake
up and have found a music video by a somewhat well-known rapper.
[demo with music]
>>: So that was cool.
>>: [inaudible].
>> James George: Right. Well, I don't know if you guys get the Max Headroom reference.
>>: Yeah. A lot of people didn't get that, but so the first hard drive that had any of the 3-D
scans that we were taking was called Headroom and the icon for the hard drive was a picture
of Max Headroom, so I was super excited about that reference, and I'm glad that I can make
videos for Eminem without actually having to make videos for Eminem. So I was excited about
that happening. So it's sort of a culminating moment for the toolkit.
Now that was the experimental talk. Thanks for trudging through it with me. I hope it was
interesting. And I want to talk a little bit about the projects that I've undertaken here so that
when you're around the atrium and people ask you what the hell is this you can may be given
somewhat of a more educated answer for where I was coming from in making these projects.
So the first thing I'm going to talk about is the Wall Queries Project which actually it was
inspired by in a large part of this artist, Sol LeWitt, from the 60s. And he was a conceptual
artist, and he would create descriptions of ways to fill an entire wall with a drawing that was
just boiled down to a sentence; and then instead of executing the drawing itself he would give
it to a drafts person, as he called them, or several drafts people, often art students, to interpret
and create the drawing. And there was a lot of psychology and play built into the way that
these drafts people would interpret these somewhat obscure sentences and the result would
be a drawing and they would be different every time. And in his way of looking at it you could
actually destroy the artwork, the drawing in the end and it wouldn't matter because the art lied
and lived in the concept; and the concept is the machine that created the art and it was, the art
could be in-permanent.
And I like this idea a lot. Now searching through my experiences in working on the Internet and
working with different artists and people who use image, found imagery online, you often do
something very similar which is you go to a search engine, you type in the word for something
you want to look at, and you're given a bunch of images back that reflect in some way that
search. So I thought why not sort of try to match up this wall drawing concept and take it to,
make the draftsperson the search engine that sort of unwittingly contributing people of the
Internet?
So I started with, in the Sol LeWitt tradition of very solid geometry searching for things like
squares, lines, triangles and would take all of these images to create these archives. I was
actually here I was actually working with Bing, and they were giving me just huge text files full
of image URLs that I could scrape online rather than having to write against their own APIs.
And I would take it into a program and run it in Open Framework, then I would sort of lay them
out, and using further interpretation of writing more code, I could find ways to sort of lay these
out in a pleasing way. So this shows a little bit of how I was working. This is rainbow sort of
overly here is because I was a sorting them by hew and I was able to actually kind of create
subtle gradients that went across these drawings so that they would, so it would be a little
more red at the top and oranger at the bottom but really trying to make a drawing, something
that felt coherent out of the sort of chaos of imagery that was coming back to me from the
Internet. And in the end I produced the panels you see up there which is a single search for
squares split across seven different colors. And actually I generated a lot more imagery while I
was here. This was a search for cubes, for example, I was having some aesthetic difficulty with
all of the white space and some of the different sizes. So in the end, I worked with the squares
because they were working the best. But I wanted to produce, I see this as a larger inquiry and
a series with, you can throw anything add it, different shapes or words.
>>: So I see in the actual composition is that coded or is this more like in detail to detail piecing
together and [inaudible]?
>> James George: It’s coded. So the way I worked this out was I, actually first time I just
printed a few of the images out and started arranging them by hand. How would I organize
these if I had to do it by hand? And the way I find myself doing it was I would take the space
that I knew I needed to fill, so the size of the wall, and I would start laying out the biggest
images first and then I would take the smaller images and kind of fill them in trying to overlap
the least, so I wrote code to do that essentially. It sorts them from largest to smallest and then
it tries, it randomly chooses places until it finds the place it’s overlapping the least underneath
it. So once the amount of imagery exceeds the size of the space then you get this complete fill,
but where they’re generally respecting one another's position but it's not on a grid. It's not like
a bin packing in an intelligent way, but it reflects the way that I would naturally like to do that
layout. And then adding this aspect to it helped me kind of created a gradient. It limited where
you could throw those things based on the actual color content of the image to create more of
a wash.
So this is the final layout for outside, which you can see, and find lots of fun stuff. And then
once they were printed I found tons of images in there that I had no idea that were in there
which was exciting. So that's Wall Queries, which is outside. The second was I was working
with Charles Loop[phonetic] and his system, and it's called Grip, and this is sort of a screenshot
from the capture. We filmed these two dancers from Seattle, and I worked with choreographer
Alicia Ghostee[phonetic], and for a long time I had this image in my head of mutually sustaining
poses. So people, different ways of holding one another up and to the point of if either person
let go of both people would fall, and just sort of trust intention that you can put into someone
was very metaphorical and for me he felt a lot like the process of sort of engaging with things
and change and letting go and the inevitability of working with technology too. But it’s a
physical metaphor.
So I just have a few work in progress screenshots so you can kind of see the evolution that I
went through. So I started with, this using Charles is code and processor, able to build a 3-D
boxilization[phonetic] out of that. Now Charles had built it originally in a very controlled
environment with a green screen, and our environment was much larger, and we didn't have
the time or resources to build a green screen out of it, so we tried to do a luma keying effect
and bring in other cameras for the color. Now in the end, for aesthetic reasons, I went with the
just the black and white, just the figurative data, the 3-D data. And in order to, can you turn the
lights down a little bit? And you have trouble with a lot of the extremities and details. It ends
up coming out almost feeling handmade, very sculptural, parts missing. I’m as interested in the
way these systems work as I am in the way that they fail because that starts to define the way
that we look in these mediums.
And then because this is not a triangulation-based system, with the Connect stuff I'm doing it’s
all composed of triangles, this is purely points and space; it’s very dense. I was able to start
playing with, and this is the psychedelic direction I was going one late night, but I was able to
start playing with actually creating sort of sandstone feeling explosions of this texture and this
feeling. And in the end the rendering started take on this very sculptural quality with
highlighting the lighting and things like that. And the final renderings look like this with
specular lighting and some washes and gradients and shadows to make them feel very spatial.
So both of these projects are outside. Hopefully you can go see them and check them out.
There is a slight bit of interactivity on the columns that when people walk by they’ll trigger the
people to fall and disintegrate and then they'll be replaced by another stance. I'm very
interested in the fact that these columns are actually architectural features. There are loadbearing, and so to put sort of a frail technology underneath that hopefully is suggestive. Not
that your technology is frail, Charles, I didn't mean that. I mean just nascent in formation.
And then as a final intervention I decided to go through with an image research project that I
had been pursuing while I was here which is an archive of actually hand-picked tattoos from
people bearing logos of technology companies. So first I was searching for tech companies,
actually I was inspired to see this. One of my first days coming here I saw a woman with a
Windows 8 tattoo. So the new kind of the slanty logo, like really prominently displayed on her
arm, and I was like wow, why did you get that, in my head. I didn't ask her. I should've asked
her. And you start to think, well Windows 8 is like, she's outlived 10 versions, 8 versions of
Windows right now. But there must've been some formative moment with the release of that
operating system that felt like it solidified her identity in a way to the point of wanting to ink it
on your body. And it made me think, that my first reaction to all of these images is that the
relevance of whatever thing is being referenced like EverQuest or Java all your [inaudible]
meme is going to die sooner than the person that it’s inked on. But may be the importance of
these technologies is not their lasting impact but their lasting impact on us and the way that we
carry them with us in our memories, and that in fact, it’s still people in the end that want to
bear these stories. And I expanded it to a lot of imagery that involved the desire to integrate
technology into our bodies and the feelings of being a machine, which has influenced a lot of
the way that we treat ourselves, this mechanistic perspective on life. So hopefully it's meant as
a bit of a punch line to the whole thing to lighten the mood a little bit.
So I want to leave you with one note to kind of circle back to the whole thing, for the Clouds
documentary we were actually fortunate enough to, if you recognize this guy, the 3-D scan
Bruce Sterling himself, and we asked, this is very meaningful for us, my collaborator Jonathan
and I we flew to Austin where he lives and did this extensive interview with him. And one of
the questions, this will all be out in the Clouds film when it's produced, and one of the
questions we asked him was: we are standing here with this device that you inspired, you
talked about this thing in 2010, what do you think of this new medium? What do you think of
these, this experimentation where things are going? And I'm going to play for you his response.
I'm not going to show any images although we have them. I think it's more profound just to
listen. And I'll close with this.
[interview with Bruce Sterling]
“I would describe your medium there as a lash up. No, obviously it’s a lash up because you’ve
got like two different devices, you’ve got like your digital canon EOS and your Xbox 360 and
they're on a tripod but they're like literally united with rubber bands. So clearly you've got to
data streams and you’ve come up with like some software method to like mashed them up,
right? I've seen a lot of game devices. They tend to have pretty short lifespan. They're built
with a very definite kind of planned obsolescence involved. So, you know, a device like that has
basically got the lifespan of a pet mouse and the one on top of it has maybe the life span of a
cat. So I'm kind of happy to see them here, but I've seen her brothers and sisters and
grandmothers and grandparents and they're all in a gigantic radioactive black plastic heaps
somewhere in rural China. They really do have the lifespan of hamsters. So you just have to be
aware that you’re like the hamster herder. Just somebody who's 25 years old today and you're
like an XBOX hacker. You're going to be like living 50 years from now, and it's good to sort of
understand what it is that makes you happy about it. Why do you want to engage with it?
What is it within yourself that's going to enable you to get out of bed 50 years from today with
a kind of undimmed enthusiasm, and that's a very valuable thing to have. Few creatives get it,
really. But the kind of key to it is like finding it within yourself to give, not finding it within these
devices, services, specific social situations, pieces of slang. You have to find this sort of inner
wellspring, and it requires a kind of self-understanding that not everyone has and that's not
easy to get.
>> James George: So, thank you for listening. Now I'll take questions if you want to ask them.
Yeah, pretty dark in here. Sorry.
>>: I think you mentioned a little bit about the aesthetics of the composition of the film linking
in some of the RGBD videos. I wonder, I think it’s obvious you spend a lot of time planning and
repairing some of these things or at least setting up rails to give you the interesting perspective
like the one where you talk about point here, point here, and it spins around. Basically you use
it as a narrative tool. If this were to become a little bit more mundane, like every day, do you
feel like it would be nearly as exciting to watch Newscast in this medium where there's no
movement whatsoever? Like is the aesthetic kind of very integral to the story delivery there or
do you feel like the medium itself is compelling?
>> James George: I think it's very important that it's integrated to the storytelling. And I don't
in any way suggest that it should be ubiquitous or used for like Newscast or things like that.
Now I let other people suggest that when I release it Open Source. And I think that's kind of the
provocation there is how general is it? It's actually a genuine question to the world. And that's
why I release it, like where will this go? I didn't think that Eminem would want to use it for a
rap video. Like that's absurd. But it’s also integrated to the story there. So I think the thing
that I'm constantly coming up against and discovering is it's used to talk about a very specific
feeling in the culture right now about sort of being immersed and overwhelmed and integrated
with technological systems and the aesthetic works really well for addressing those issues. And
it's fun to use that tool and track the tool as a way to, a lot of people are playing with this and
exploring that. But yeah, I think in no way is that a video camera in the general sense of image
making for anything.
>>: I think there's always a very interesting [inaudible] between the technique kind of what it
opens up verses the composition that's there. Like you look at something like waking light>> James George: Right. Super relevant.
>>: It took an insane amount of effort at the time to do it, right? It would have taken Herculean
effort 10 years before that, and now maybe it would be kind of trivial to do, but the art and the
stories that they tell and endear and I wonder some of the things, especially putting it out like a
toolkit maybe there are artists who are thinking in these kinds of compositional thoughts are
now enabled to be able to do that because the toolkit’s available and it’s their story in
whatever composition they have that will endear just like in your Clouds piece. I think that yes,
there’s some particular strapped together thing that you made [inaudible] implied because
that’s a lifetime [inaudible], but in the end you used it to capture these really interesting people
telling these really interesting stories, and [inaudible] was saying to kind of do composition
around them in a way that enhances those stories and that's what's going into your>> James George: You answered his question, right? And these are the ways that I try to
answer that provocation is that what are you, you are engaging with the current time, the
people around you, the way just creativity in general and thinking novelly[phonetic]. And I
think the danger is that you get recognition or attachment to a certain process or certain device
and then it will, you will die with it. And I think he wasn't indicting us. It was a warning from a
mentor in a way, but I think yeah, I think you said that, you answered that question very well.
>>: Do you think he would've been more impressed if you had gone in there with like a Latrobe
camera? Because he was talking about doing the microphone and [inaudible] and I was
thinking Latrobe all the way.
>> James George: Yeah. I don’t think>>: Yeah, that's exactly what I was talking about because you’re capturing all the photons>> James George: Even if that’s exactly what he was talking about he’s a dissolutioned prophet.
He wouldn't be impressed by anything. And I think that's kind of his stance. I think he is very
impressed by all of it. He's been tracking all of our work and has been very open to
conversation and was down to interview. I mean he likes that, I know him enough that, his
father was an engineer. He really envies engineers as an author. And one of the questions we
asked him was what software do you, write? And he's like, something boring like a word
processor. And even though I don’t think that's necessarily boring but he was sort of selfdeprecating in that way. So there's this relationship where he has a larger view of the field and
has the benefit of perspective and time, but where he still really respects the people that are
out there on the ground making things.
>>: I sense he was sort of diminishing of the whole thing. I mean, is that a form of selfdeprecation [inaudible]?
>> James George: No. I think he was diminishing it as if it was premised on the fact that it was
the future or if it was this camera the future. But I think his statement is really that there is no,
that thing doesn't exist as a single thing. It's going to be instantiated on all these different
forms but it's really that idealized thing is just an asymptote. It's just something that's in our
mind and that we are constantly working towards and understanding why we are working
towards that is important.
>>: I have a question for your abstracts. I'm trying to get it out and see how it goes. So around
the notion of creativity being a very kind of human idea or human behavior but a lot of the
[inaudible] here it kind of offsets between the creativity of the machine. Is it possible
[inaudible] machine has perspective or has a process or is it more on the other side of these are
digital prosthetics for us as we create in new ways to do that and look at your thoughts just on
that range.
>> James George: We talk about, you know, computers are awful poets but they're great
collaborators. And I see it as, personally in the way that art that I see successfully uses
machines as a prosthetic for the artist creativity and it still needs to be the person who is talking
or thinking or playing and it allows you to do things that would otherwise be impossible by any
other analog methods; and when you try to get the agency over to the machine and
anthropomorphize it and say that you, here you make the art because I don't want to or then I
think that people who, that art is harder to relate to. It tends to be very sort of starker or
something's missing. And I think that there is a tension there. But you’re constantly, when
you're using these prosthetics they’re always sort of trying to take over. In the same way it’s
just a material in that you can't let, a famous Sol LeWitt quote is that an artist who is in love
with that their material can't make good work with it because if you love it you can't see
through it and so you're constantly just putting the material out there on display and being like
look at this, this is great. It doesn't translate because other people aren't in love with that
material in the same way. But if you're using it to go through something, actually if you hate
your material you're going to use in the most sparing way possible to get at what you want.
And I think that would end up making better work. So this is a relationship that I'm having
because I definitely was in love with these three scanners now maybe I hate them in a way.
And that's starting hopefully, that tension is starting to make me a see through them a little bit
better, and that's my challenge.
>>: [inaudible] art wise I feel like when a certain technology appeared they can devalue a
certain type of art. So before, like photorealistic painting was really respected, it was incredibly
important thing. And then it’s like as photography quality got better and cheaper it was like
well that’s not so great. And now like, you know, I did this oil painting and it looks a lot like this
photograph, and that’s like yeah, but there are these nice Photoshops alter your stuff that will
be almost the same thing so then like 3-D and you showed the kid with the sculpture of himself
and it’s like does that devalue the sculptor who would've made that carefully out of clay?
>> James George: I mean, look at Sophie’s work. I mean, make or buy a3-D print of
somebody’s head next to Sophie’s sculptures and tell me which one is like that’s using the same
ABS material, right? I mean she's being very thoughtful about, and I think what happens is
when a medium is introduced that competes with another one it starts to shear off utilitarian
aspect verses the expressive aspect of it. And so it's like yeah, painting is not very useful for
crime photography anymore or like personal archives, but yet, and it defines itself as its
expressive side of it and painting became so much more expressive when it was defining itself
against photographic utilitarianism.
>>: [inaudible] necessary for that art to survive to harness that expression. It can’t just be
photolistic[phonetic] anymore because now there’s a competitor, so to speak that’s>> James George: And I would say that is only does good for the variance of things to
experience in the world. Diversifies it.
>>: Did you think when you did these toolkit that this [inaudible] wide range of people and then
they all use it and everything that they’re saying that if you're [inaudible] independent art is
your own way in solving basically [inaudible] problem with your different solutions will look
different.
>> James George: Well, that's a really pointed question I struggle with a lot, especially because
we embedded very specific aesthetics into it like the way the wireframe works, the way the
overlays work, the blending modes, the depth, the field, it's not just the data as a stream. And I
think that the way I’ve come to terms with it is when you release your aesthetic openly you are
able to speak much louder but you lose all control of what you're saying.
So it's this weird tension where you're actually gaining, all these projects anyway point back to
me. Like when Alejandro released the sex tape with his girlfriend in RGBD he sent me a note,
he like published it on Twitter, sent me [inaudible]. I’m really happy. Thank you so much for
helping me make this. Can't wait to work with you. I was just like I don't know, totally didn't
know what to do. And now I think it's actually one of the best moments in that whole
experience because I think that the notion that you're in control or that you own an aesthetic is
not a productive one if you’re really genuinely asking questions of like the visual experience in
the world. It may be productive you're trying to like canonize yourself as an author and work in
that way.
So if, you'd be the only one who could create something like this, but in the way that I work
that doesn’t, it’s not something I'm interested in doing, and I find a lot more creativity and
expressiveness in just releasing things where they go. And ultimately, when things start to
change and are made, the stuff that sticks is not about how it looks, coming back the story side
of it, and that you're actually transform forced to transcend your medium even a little bit more
if everybody can make things that look the same then it's your use of it that comes through and
makes it meaningful.
>>: What kind of folks did you interact with while you were here, just teams or groups?
>> James George: Yeah. I mean I interact with [inaudible] and Andy and Charles, but actually I
worked with the Connect team to create a new wrapper for the Connect for Windows SDK
because I haven't ever used the actual Connect SDK for this film making stuff because it had a
lot of limitations that made it not work for this particular application; and so I was able, they
were really receptive to me being here, and we created this new layer that basically makes it
accessible for people who are not seasoned developers, they are more entry-level or tinkerers,
and that’s a thing that I really, I mean that's really the thing that I stand behind is that
programming should be accessible to people who don't considers themselves programmers,
and you’ll get a lot more interesting technology in a world that has accessible technology. And
that’s something that, I think doesn’t always, a sentiment that's not always brought up at
Microsoft when they're thinking about their developer tools is they kind of have this
expectation of a great level of knowledge of the people that use their tools.
>>: So they are inherited in definitions, expert level, this whole kind of emerging nuances, what
developer means [inaudible] fascinating to me. [inaudible]. So did you find as you went
through the different groups that there was different perspectives>> James George: Everyone was really open. That’s the thing is also a place where nobody
seemed set in their ways so I was able to have immediate effect and change and influence. So
that was cool. Yeah.
>> James George: Cool, thanks for the questions. [inaudible].
>>: Thank you.