Rick Barraza: Thanks, everybody, for coming. This is the Ambient Creativity Hackfest, which we've
been going out on over in Building 20. It's one of the first for Microsoft.
First off, my name's Rick Barraza. I'm in a group called Quality in Experience Design, a developer in
evangelism. Been working with a lot of our partners for a while now. And this Hackfest, we've really
been focusing to help have these conversations and enable these tool sets for people who tend to be on
the envisioning side of things.
A quick roster of everybody who's here, and we have some really special slides today. And tomorrow
we will be at the garage [indiscernible] code mixer for kind of the second half of this talk. So if you
can make it to that one, that would be great. So we have some really interesting things to show and
kind of a thematic story to start the day. And tomorrow we can end up with pizza and beer at the
garage, which is a great way to end any two-part talk. But from the Hackfest, everybody -- most of you
in the front row and spread out throughout the audience, but we've invited friends from Digital Kitchen,
who have been doing a lot of great work with us, traditionally as Microsoft, but they haven't interacted
too much from the engineering side of things. And we're really trying to address that issue of what
does it look like when design and engineering and the arts start mashing together. So that's what this
Hackfest is all about. So we've invited them to come in on those conversations.
We have Stephen and some -- well, actually the Digital Kitchen guys, can you kind of raise your hand
so we know -- they're sitting over there. Where is Al?
>>: [Inaudible]
>>: Okay, good. So we have people up from Portland from Weiden+Kennedy, specifically The
LODGE. And they have a great presentation today. And Genevieve Hoffman is right over there.
Michael Szivos will be closing up on architecture and how computation is affecting some of those
mediums. Anna, also if you can waive; you'll be talking today. James George is a bit of a
homecoming, I guess, and we'll start off with him. Joshua Noble from FROG; and Joel Pryde,
Stimulant; and Lia Martinez, who is now working at Microsoft on the Internet of Things.
So when we started the Hackfest to when we started pulling off -- come to find out one of them has
actually starting working with us, so that's kind of good. So there's an insider track there, if we want to
go that way.
And again, my name is Rick Barraza. Before we start off on the slides, I think we'd like to first loop
back with James. A lot of this kind of started -- these conversations started two years ago or so when
you and Golan came to visit. Can you give a quick summary of that initial visit, the artist in residency?
Because you are our first artist in residence; right? And how that's been going so far, specifically with
Clouds?
James George: Sure, you want me to -Rick Barraza: Sure, come on up.
James George: So yeah, this is very [indiscernible]. I haven't prepared anything. So working forward,
I met Rick almost two years ago now when I came with my mentor, a professor from Carnegie Mellon
named Golan Levin. And we were invited to do a presentation on what it would be to have an artist in
a research institution like MSR. And then there's a long history of this with things like Bell Labs, other
research institutions that have worked with people who are traditionally associated with being artists,
such as like Andy Worhol.
But there's less of a tradition of artists who use code as their primary medium and work outside of
technology companies, or even, you know, research or academic context that are actually in the wild
making their work in cultural capacities for institutions or freelance, interfacing directly with a place
like MSR.
So we came and we talked about what that could be like. There's obviously a lot of resonance between
researchers and people working in non-traditional ways with software. So I got a long with a lot of
people here at MSR. And the following year I came back for a pilot program, a three-month residency.
Rick Barraza: And can you give a brief -- sorry to put you on the spot -- on Clouds and kind of when
you're with MSR here, but some of the things you're exploring?
James George: Sure. I have a MAC in front of me, but it's acting like a PC. Looks like someone's
been Googling that. [laughter] I know, a long history. A lot of white beards. Maybe I'll show some
images. So without maybe -- I can probably play this. If it won't do sound, I can probably turn the
sound off.
So this is just a demo of this project, so one of the [music] -- does that turn off? You can watch this and
research it on your own time. But one of the main reasons I was invited here is I was working on this
design fiction software project that imagined the future of photography through the Kinect censor, to
put it in broad strokes. And the way I was working with that was I was interviewing a lot of people in
this community and in the arts computation engineering community and kind of creating this
interactive experience that allows a viewer to navigate this archive of over ten hours of footage. That
all takes place in computational 3D space. So that's this project.
Clouds, it's now sort of completing it's run-through in the festival circuit. We showed at Sundance in
Tribeca in their interactive insulation divisions.
Rick Barraza: Yeah, we have a great shot of Robert De Niro.
James George: So this works on an Oculus Rift, which is a virtual reality head-mounted display that a
lot of people are talking about, recently acquired by Facebook. We've been -- actually, the first time I
touched an Oculus was here at MSR with Andy and Indian Bancko, and created a prototype of a project
to work on the platform. So we've been showing it that way too.
So it kind of goes to show the way that someone like myself, coming into a place that has access to a
lot of new technologies and new techniques, that have somewhat different intentions and different
audiences that I speak to with my work, there can be a lot of synergy there.
Rick Barraza: So just ask a couple of more questions while you have the microphone, and then we'll be
moving on. But let's actually dig a little deeper. When you started this -- because you didn't start first
with Clouds, you started hacking the Kinect; right?
James George: Yeah.
Rick Barraza: So you also mentioned design fiction, which might be a term that a lot of people aren't
really familiar with. We've had these conversations around zero-scale productions. So can you actually
talk a little bit more about how you got into computational photography, what Kinect did, and frankly
how you started hacking the Kinect, on what platforms and why you chose those platforms?
James George: Yeah, those are very leading questions, Rick.
Rick Barraza: Well, we've had these conversations many, many times.
James George: Definitely. So just briefly, my background's a mix between art and computer science.
So I studied both, actually here at U-Dub. And but I've been living in Brooklyn working freelance
since graduating, undergrad. And particularly I moved to New York because there's a hotbed of people
there working on this thing called openFrameworks, which is a C++ Open Source toolkit for doing
essentially graphics, high-powered graphics, and vision in a very, I guess, designer-friendly way,
likewise, that kind of get rid of a lot of things that make C++ inaccessible to novice or self-taught
programmers. I've been contributing a lot to that platform and really thinking about interactivity and
computer vision and thinking of the esthetics that are coming out of that.
So when the Kinect came out, to me, it was great for interactive installations, which we were doing, but
it also suggested this potential future where you have like a merger between computer graphics and
things that are synthesized like a Toy Story computer graphics and live action, which traditionally has
been more of a compositing work flow, like let's do make one and make the other and let's mix them
together. But when you start having these computational cameras, things that see the world in a
geometric way, you can see where in the future that might start -- that distinction may start to break
down.
So we started making projects that were rooted in stories or sort of science fiction, but actually began to
explore what that may look like. And never really, especially at the outset, intended to develop
anything that worked or could be distributed or sold as an application. The software never left our
computers, but the images that it generated sparked a lot of imagination and conversation around it. So
if you think the word design fiction, it's sort of like science fiction, but where the thing that you're
fictionalizing is a particular project or process in the immediate future.
Rick Barraza: How will it effect people?
James George: And you don't need to make it, you just need to make a story around it, and that can
often influence what other people who have a lot more time and energy than you will end up making.
Rick Barraza: Well, it brings up an interesting point. Because you not only need it to -- do you have
the picture of the RGBD toolkit?
James George: Sure.
Rick Barraza: So -James George: So I updated my website like three or four days ago. I'm so glad I did. So we actually
renamed it to DepthKit, because RGBD is really hard to say, and it was not that DepthKit is that easy to
say either, but -- I mean it does work. People can experiment, you can download it, you can install it
on MAC or Windows, calibrates an SLR, external video camera to a Kinect sensor, an Asus sensor, and
allows you to make this kind of science fiction neuromancer digital reality feeling video which has
become coveted in sort of a -Rick Barraza: Well, we had Eminem. Didn't Eminem use it for their -James George: It's been used in a few different places from the Open Source software. So one of the
most -- probably the most views that has been gotten is Eminem's latest Rap God video used a few
sequences of it, and we worked with a director to make sure it's working for him. But my team isn't -our collaborators aren't actually doing any of this work. Occasionally we freelance, but we're really
putting it out into the world free and Open Source and seeing where it goes and what conversations -meeting people through it really.
Rick Barraza: It's great having the kind of side-to-side facing the audience conversation. But one of
the things we talked about on this is we are seeing at zero scale and creatives leveraging computations.
When you're really at this exploring state, it kind of goes back to that Renaissance period. And by that
I mean you have to be both a tool maker, and you're very active in the openFrameworks community;
right. So you're a tool maker that makes the paintbrushes and paints, and then you have to turn around
and paint, you know, the Last Supper at that point. And that kind of Renaissance meshing of
personality, of skills, seems to happen at zero scale experiences, right; the prototypes, the concept cars,
the fashion couture, right; the things that we're doing with technology these days.
So in that area of being a tool maker who also turns around and uses the tool as opposed to saying I'm
an engineer who makes the tools, and I give them to artists who don't know how to make tools and only
know how to paint pretty pictures, I mean, is that one of the main drives from openFramework and
Cinder and processing?
James George: Yeah, I think there's a big incentive. There's a process that's developed where it makes
a lot of sense to not have a very big boundary between the user of the tool and the maker of the tool.
So you'll have some -- like Digital Kitchen is a great example where there's a software developer sitting
right next to a designer. Occasionally they're in the same body; really it's the same person. And what
that does is you end up with tools in a very general sense, and they end up -- oftentimes the user for
that tool is one person, and it could be the person who's making it. But those can be kind of those -your word for it is these zero scale. So it's never meant to scale, it's never meant to be lived beyond the
life span of the project that it was designed for. But the processes that are uncovered for working
directly with software in a very sort of fastly-iterated, without the overhead of worrying about
generalizing any of it, can be really productive.
Rick Barraza: So thanks on that. And we're going to call up Stephen. Because as we're seeing this
progression from -- [applause] So these concepts of zero scale tends to go against how we've
traditionally approached software at Microsoft. We're very, very good at transactional scale
significance. And we get small-scale-hype premium. But zero scale experiences, couture, right, it still
can be a bit of a challenge for us as we're changing our culture to move into this space. Yet everything
we care about tends to start at zero scale. It tends to start at a clay concept car. Concept cars are zero
scale; you're never going to buy it later, but it points the direction. Fashion trends. Now those we
talked about from the openFrameworks and the academic, the pedagogy there, very focused on
academia places like MIT, the media lab, ITP, the places where Genevieve's going to be talking about
as well. The academic is still very focused on exploring. Stephen now works for Weiden+Kennedy
and he's going to talk about how do you start bridging that gap between zero scale, where you're
exploring the medium, to actually turning it into profit. That's Weiden+Kennedy. So can you get your
slides up?
Stephen Scieberl: Okay, can you hear me? All good here.
My name's Stephen Schieberl. I'm a creative technology lead with The LODGE. The LODGE is a
department at Weiden+Kennedy. And this is the mission statement of Weiden+Kennedy right from the
website. "We are an independent, creative-driven advertising agency that creates strong and
provocative relationships between good company and their customers. We believe that it doesn't matter
where, how, or in what medium an idea is expressed, you still have to start with a good one." So it's
really a sort of -- it's an ad agency, but we have a mission statement closer to what you'd expect to how
an artist would communicate with their audience.
The LODGE is the creative technology and experience design group that sits in WK's creative
department. And so it's one thing to note is that we are in the creative side of it. And the way that we
fit in is we use code along with art and copy to concept, prototype, and build experiences that solve
business problems in a way that feels like magic.
So we're in the a production department. This is what you usually see in ad agencies is a creative
technology department. Usually just means programmers that have good design sense. But really
what's happening is traditional creatives are coming up with ideas and then they're throwing them over
the wall to the developers and having them make things. We're a creative department. We are a group
of people that understand technology. We code, but we also -- we use this as an artistic medium. And
therefore we're specially positioned to understand problems unique to some of our brands.
This is something you hear a lot from programmers and ad agencies is we need to get involved sooner.
There's this problem that happens of your programmers are getting a project in and being told to make
it, and the first thing they'll say is this is not a good idea, this is a bad idea. If we got involved sooner,
we could have fixed some mistakes along the way. So then the question comes up how soon should
programmers get involved? And that's sort of what the impetus for The LODGE was originally. It was
trying to give us a little more autonomy, a little more freedom to get involved in the process sooner.
And what we've been finding is that especially when a client is a tech company or it's related to
technology at all, you want creative technologists right at the beginning. We're usually able to find and
solve problems that traditional creatives aren't able to, or even traditional planning and strategy folks
don't notice the same way we do.
Rick Barraza: Not to take you off your [indiscernible] but so it doesn't get missed. This is actually the
inverse of a lot of what happens at traditional software companies, because you're a creative agency,
and you're arguing that developers need to be brought in as soon as possible. And the inverse story is
in traditional software companies we're trying to bring design in as early as possible.
Stephen Scieberl: Yeah, and to back up a little bit. When we built The LODGE, we had -- we actually
built the department out of people that were already there. We pulled people -- we kind of looked over
the agency and found all those people that were programmers. We've even pulled some people out of
IT, some designers, some UX people, and we've formed this team of just if you can code, if you know
technology, you're in. And we were expecting to have essentially what is the second programming
department that did better. And what it's turned into is far beyond what we could have thought. We've
really been able to dig and to look at our clients and find real problems they're having and find
solutions for them. We're beyond just making ads.
And one of the things that we found is that by doing this more regularly, getting this team in front of
the clients earlier on in the process, we can find out where they do belong or if they belong at all.
Sometimes a client's problem can be solved with a great TV spot, which when -- there's plenty of.
So the best way to describe what we do is we embrace technology as the best tool for involving the
largest problems facing our brands and our customers. So we don't make ads, we solve problems. And
we don't have a process, but we have principles that guide us through to these solutions.
And I just want to walk through a couple of the few other things that we've done. Turbo Tax is a client
we started working with recently. And usually a department like ours and a talk like this would just go
through and show you some execution, some projects that we've made. But the work that we've been
doing has gone well beyond that. With Turbo Tax, we've been looking at essentially what is a tech
company's platform through the creative tech lens. And we are changing the way that people do taxes.
So it's not just coming up with an idea or an app. The message that's core to Turbo Tax has been it's
amazing what you're capable of. And we have been looking at Turbo Tax and we're continuously
inventing new ways of resolving complexity in taxes and the way that you do taxes, and this is a very
fundamental thing in our lives. So to be able to go and -- to go to them and then have the ability and
the autonomy to say, hey, we're noticing some real problems about your technology, or even not so
much the technology platform, just the whole concept of taxes. Our department, with this -- with our
perspective of spanning design and creativity and programming and everything has had this look at it
which is really changing the way that you interact with your software.
Weight Watchers is a client that we acquired recently. And this is another one where with food-related
health problems on the rise, we've had this opportunity to use creative technology to solve an important
culture problem and a partnership with a client whose business is based on it. So this is one of those
things where something as fundamental as health, from our perspective we're able to look at it and
work on a platform that is going to change the way they do business.
More specific example is this recent project called Happy Calories. So Coca-Cola wanted to add their
voice to the health conversation. And we worked with our traditional creative team and Coke Global to
develop an idea and an execution that addressed the topic in a direct and honest way, but still seen
through the Coke happy filter. So we made people earn their Coke and have some fun doing it by
having them ride this thing.
So this is a big bike that we put down in Huntington Beach. And in the back there is an insane glorified
Coke machine. And a Coke has 140 calories in it. And so we had this idea of to get the message across
that there are 140 calories in a Coke, maybe you shouldn't just sit there and drink them all day. You can
have a Coke if you take the time to earn it. So we put this bike together and we turned it into an
exercise bike and we were actually -- everything on here is very handmade. We've got some
monitoring equipment on there so we can accurately weigh the person and look at how fast they're
riding. And that's used to send a Coke through this crazy contraption and things go down slides and get
pushed around by roller skates and go auger lifts, and then eventually it's shot up -- when you hit 140
calories, I think you're shot into the machine, an inflatable guy comes up and people have a lot of fun
doing it.
This is a really interesting thing that's come up a few times now that it's almost a service that we have
just found -- we found ourselves doing on these bigger projects. Nike asked us to do something big at
South By Southwest. And we decided like what's bigger than the sky? So we wanted to take the sky
over. But we had less than a month and really limited budget to make a projection screen the size of
four city blocks hovering above Austin. Given those constraints, we didn't want to just dive in and then
find out later that it wasn't going to work. So we created this floating projection simulator. And
something that's really interesting about this is that we actually tied all the parameters to hard costs. So
we were able to figure out how much helium goes in the balloons, what the size is so we can do all the
physical dimensions. We call these parameters, see what this thing looks like. Put in the all -- all the
parameters about the projector and see how far it would actually project. Would it be bright enough?
We can change the view so we can see this is what it's going to look like from the ground. And then
eventually, as we're playing with these parameters, we can see what it costs, what it's going to look like.
And we're able to look at this and say this is actually going to cost too much and take too much time
and we ended up not doing this project. But it's kind of an interesting thing that happened that I haven't
seen in another team before. And we've made a few of these now and it's been a really helpful thing to
start doing.
Something else we do at The LODGE, we're huge proponents of Open Source. We use it, we
contribute to it, we make it. We're major contributors to the Cinder creative coding framework. We do
a lot of Django stuff and make a lot of other tools. And we found that it really improves the tools that
we use and the relationship with the community that uses those tools.
We've been working with Microsoft a lot on the Kinect stuff for a while. And this is a screen shot from
one of the samples that we include in our Kinect Common Bridge, Rapper For Sender. It basically
takes your depth image and blows apart into particles, and it's really pretty and fun.
And we've been working with Microsoft on this and the View 2 sensor and getting these libraries out to
the world. And we get a lot of people contributing back. And it's great because a lot of people in this
community are adverse to ad agencies and our type of business. And it's -- this is kind of a good bridge
between the art community and the commercial world to say like we're not such bad guys; we can all
be friends.
Another Open Source thing that we've put together, this is a really fun thing. It's called Composite. It's
a service that enables multi-screen experiences in HTML 5. And this is pretty significant, because it
gets you over a hurdle defining a scalable model for distributed messaging that works on multiple
platforms. What that really boils down to is penguin curling across a bunch of devices. So we have
this game.
And some things that were really cool about this is we got this Open Source library to a certain point.
And we went to Coca-Cola and we showed them this technology and we talked about how we wanted
to use it, and they actually helped us finish funding developing the library. So we got one of our -worked with one of our bands to develop an Open Source library, actively made the funding on it. And
as you can see on this, it's a very simple way, if you line up some devices, swipe across them while
you're in a browser, it can be any browser, and then you can play a game all the way across them; you
do curling, you sweep as you go down.
Rick Barraza: Steve, actually one question on this, because it's something that's been coming up in my
conversations. Are you finding this more and more at Weiden+Kennedy, and hopefully through other
agencies, that if you're working and using an Open Source library, you start negotiating in the contract
as you do work for them, hey this part that we do to enhance Open Source library, we do want it
covered, because you want to give it back to the community?
Stephen Scieberl: Yeah, absolutely. And that's the thing, we have to be really transparent with our
clients about what we're doing. There's a lot of times that we like to do our own R&D and we develop
libraries for the community, and then we end up -- we usually focus on things that we think we're going
to want to use. And when we go to our clients -- we're very transparent about what we're using.
Sometimes they don't want us to use them, to use Open Source libraries; instead they want us to
develop everything, they want to own the code. But we have really good relationships with our clients
and they trust us with technology, for the most part. And we've been finding more and more that
they're open to being part of creating these things and getting them out there in the world.
Last project I want to show you is something that we did internally. There's a Wiedenism called "the
work comes first." And when we were asked to create a permanent installation for these words, our
response was to make it a demonstration of the ingenuity, creativity, and patience for tedium that goes
into truly great work. So we were originally tasked with just putting something on the screen with a
projector. And we were like, no, no, no, we don't want to do that, we wanted -- we want to really
demonstrate that our newly-formed department knows how to make things and can figure stuff out. So
we made this guy -- I don't know if I get any sound here. [indiscernible sound] So this is a sign that is
made, it's basically a giant wall, it's a pegboard wall, something you'd hang your tools from, and then
we have it backed by about a thousand solenoids that are attached to these painted screws. And so
when you look at it, it just exudes work. It's a piece of hardware. It's something that -- it's parts that
you would expect to see at the hardware Store. And we really -- and this is really built from the ground
up, in-house, with like very little sourcing outside of our work. These are -- we custom designed and
had these circuit boards printed so when this -- complete with fancy little blue LED lights. And you
can see all the wiring that we did and soldering. So it's something that it's a piece of technology that's
really pretty, but it's really handmade. And it kind of rounds off what a creative technology department
can be. So rather than just creating executions, we do everything. We went and grabbed all of our
programmers, we got them altogether, and what we found is that they can do everything from making
stuff like this all the way down to digging into the deepest problems that are happening inside of our
brain and finding the solutions for them.
Thanks. [applause]
Rick Barraza: We're seeing this thread to continue to emerge from this new breed of engineer and
sometimes -- we don't have the labels for it. I think that's one of the challenges that we've talked about
quite a bit. I don't know, what's on your business card? Does it say wizard?
Stephen Scieberl: Creative tech lead.
Rick Barraza: Creative technology lead. We have design engineer, we have art engineering, right,
computational photography, computational architect. This challenge that we don't even have a label -Joel, yours is -Joel Pryde: I'm a technical lead. I can add creative to it.
Rick Barraza: Technically plus creative. Slash. So the fact that we don't even have a label for it makes
it really hard to hire, makes it really hard when you write your resumé, makes it hard, when you walk
into a place and say are you a designer or a developer, and you say yes, right, then nobody wants to
believe you if they've been in traditional software.
But Weiden+Kennedy, top global brand -- I think this brings up the question, how is it effecting how
we define who developer is, including this diversity, as well as how do we also train for them?
So our next slides we'd like to bring Genevieve Hoffman, who's worked as an educator and a
practitioner, and just all around everything. Pretty impressive. Let me just load up your slides, and if
you could talk on the educational aspects of it, that would be -Genevieve Hoffman: Yeah, sure. Yeah like Rick said, it's hard to sort of define what I do sometimes. I
call myself an artist, I call myself a designer, creative technologist, sort of all of these creative coder
different roles. And also really passionate about education and sort of how -- because I am not a
technical person. Like I came from the arts, studied sculpture and painting. But then sort of fell in love
with immersive, like, artwork. And that's how I sort of came to this world. And so one of my passions
is also teaching other people because it's become so pretty much easy to do that now, how to do that as
well.
So to give you a little bit of background about my work, one of the ways I'm also supporting myself
right now is I have a residency at Autodesk. They have a digital fabrication studio, and I've been doing
some experiments there with 3D printing. Obviously they kind of let us go to town on all of their stateof-the-art 3D printing machines, water jet cutters, that sort of thing.
So this is a series where I'm sort of digitally creating landscape out of financial data. So you can kind
of see those logos from the Dow Jones Industrial Average. But spatialized, essentially. There's a side
view. And I've been working sort of like experimenting with different materials, combine it with
projection. I'm not really going into the background, but basically I'm taking raw data, sort of
extrapolating it into code in the way that makes sense for what I want to create as a metaphor, not
necessarily be readable to, you know, someone trying to invest. But this is a little bit of my process.
I've been working on making digitally fabricated ice cubes with this same sort of process. So there you
have the relief that's been 3D printed. And then I vacuum form a tray around it, and then I make little
ice sculptures that I photograph so they can live on for posterity, but then they sort of melt over that
process of time.
This is from a recent installation. I've been really interested in the arctic and Greenland, so I'm sort of
like creating objects that came from an expedition but don't actually exist, but they've sort of walked
this weird line between what is science, what is data visualization, and what is art. So I'm hoping that
they sort of ask people to understand what this process could be. So I made an ice core out of resin that
sort of has this lit quality inside. So it's not real, but it sort of evokes what's actually a real piece of,
like data visualization -- or data within the Earth, which I think is really meaningful.
But to backtrack a little bit, I got my start sort of in this world by accident. I finished my undergrad
career in the arts or education in the arts. And I moved to San Francisco and sort of through an
accident of Craigslist and just knowing who she was, I ended up assisting this artist name Camille
Utterback for a number of years. You might know her for this piece Text Rain, which was sort of like
one of the first sort of interactive installations to get a lot of recognition.
Like basically a camera; it's that's little black hole in the center of the wall, it's pointed at people. She's
able to sort of recognize that you're a person because it's a white-lit background behind you. And so
the text falls from the top of the screen and lands on you and you're able to read it sort of using your
body to do this. So I thought this was really amazing.
And sort of over the course of -- I was helping her out for four years, learned a lot about her process.
She was also a self-taught programmer, which is kind of amazing. So she was -- before there was
openFrameworks or anything like that, taught herself C++ and how to do camera vision on a pretty
low-level scale and was able to do this in the 90#s# when not a lot of people are doing it.
And towards the end of my time with her, we pitched a public art project. She started doing much
more interactive installations in public spaces. So for art projects and that kind of thing. And this was
a project that involves LEDs. So it's sort of -- you can think of it as a visual light organ. Like there's
these censors in the handrail that has a pallet of light cycling through it, and when you tap it each one
of those sensors is mapped to a hanging column so you can kind of play the light pattern that's going to
animate up there. I should have a video. But I was like, this is amazing, it's like a physical experience
in space. But wouldn't be possible without programming.
And I found out about the grad program I went to through Camille, because she was actually an early
alumna. So after I left San Francisco, I moved to New York and hung out at ITP for about two or three
years. And ITP stands for the interactive telecommunications program. And it's within the Tisch
School of the Arts at NYU. And it was founded in 1973 by this sort of visionary woman named Red
Burns. She was really -- she believed in like the power of democratic media. She was really involved
with the personal PortoPack machine, and public access television essentially that was sort of like when
there was only network media. That was like the biggest sort of cutting edge way for people to have
some expression. So she formed this program. Telecommunications was like mainly the biggest form
of technology at the time, but the name has stuck around.
But obviously like the technological fads have changed. And yeah, it's basically a two-year program.
It is within an academic institution, but I would say it's very antiacademic. They sort of take people
from all walks of life, artists, engineers, fashion designers, advertising executives, pretty much anyone
who wants to dedicate two years to being like very hands on with technology and sort of thinking about
the future of what that could be in a very sort of playful environment.
So I was there for two years as a student and then another year as a researcher and it was fabulous. I
really enjoyed my time there. It's where I met James, met a lot of people, Lia. This is a project -- I'm
just -- this is one of the first projects I made at ITP. To listen to old tapes. But you had to sort of break
them apart in order to listen to it. So I have sort of like an affinity for lost forms of technology that
won't be around sadly for the future. You did have to destroy the tape in order to have it be an
installation. So I'm probably quickening that process. But it was a fun project.
This was also a fun one that I worked on there. This is a project called Metro Change. And we were
working for a while to get it sort of somehow integrated into the MTA. But the idea was -- I don't
know how many people ride the New York City subway system, but they're sort of designed to have
like small amounts of value left over on your cards. So we were like wouldn't it be cool if you didn't
have to like just toss that or add more money to it or something? Like what if you wanted to use that
money for some bigger purpose? So we prototyped a system that would read the value off your Metro
Card and then ideally pool that money together in order to donate it to some sort of larger, more useful
thing. And for a variety of reasons, mostly because I think it's like a $55 million budget that that
translates into for the MTA, so they actually take that lost money into their operating budget. Didn't
fly. But it was actually really fun. We like hacked the Metro card and figured out how to read -- how
that bit information was flying. It was only successful when we used like a 30-year-old card reader,
because the ones built for credit cards was actually too fancy, so we couldn't get the right information
off.
ITP has a partnership with the UAC, the interactive corporation something. They're an internet holding
company, and they have this weird building on the West Side Highway. But the cool thing about their
building is they have this like 120-foot-wide screen in their lobby. So I was able to sort of work there
for a while and develop content for the IAC. So this was a project where I'm using -- I'm pulling in
photos that kind of go up on the Daily Beast. I think they have their like year-end photos, probably
from Getty Images or something, and just displaying them, sort of breaking them apart, they kind of
animate around and come back together. So just sort of fun things that I was able to do sort of in the
service of they just wanted something fun to go on the wall.
Last summer I was also teaching a two-week summer school course at the CIID in Copenhagen. And
Josh can probably speak to that, or program a little bit more, because he spent a whole year there. But
it's kind of an amazing sort of ITP like place in Denmark where people come together for I think it's a
year-long program. Normally this was the summer school, so it was two weeks. So you have people
from -- some are professionals, some are sort of trying to figure out, do I want to do interaction
designer, what would that job look like? And so our -- I was there with another colleague. We were
teaching sort of a very quick introduction to physical computing, but we were doing it from a
framework of trying to teach about bio-mimetic design, so design inspired by nature and technology
interfacing with nature in specifically. So this was an assignment that I thought was pretty fun. It was
like -- I think we taught them physical computing, just like a crash course, and then we had them go
outside and do an observational exercise. And some of them were like coming from design
backgrounds, so they were sort of used to design thinking and thinking about a users needs and what
you might need to deliver to that person. And so we were kind of trying to fit within that framework
but then get them to think a little bit differently than maybe they were comfortable thinking.
So we were like what would you design -- what would you design for a non-human user? Like, what
would that experience be like if it's not someone that you can sort of get in their head? Like, would it
be a plant? Would it be a bird? Would it be a pigeon? We were sort of thinking along those lines. And
we had an example of like here's a very simple arduino project that measures moisture. So how would
you sort of hook that up to a plant? Maybe the plant can tell you like when it needs more watering,
something like that. So it was pretty fun. They came back with like crazy ideas. I think this was like a
better pigeon gathering area. There was like a UV light because they don't migrate and they need better
light in the winter to have that sort of southern vacation experience. There was like a plant that listened
to music, of course. A tree that could tell you its name or how old it was, something like that. And
then like a little prototype ATV robot. But it was just very quick prototyping and kind of amazing to
see projects that just happened really quickly, maybe because they weren't used to thinking along those
lines.
And just to wrap up, in addition to working in Autodesk doing sort of my own work, I am also working
on a public art installation sort of on my own. Or with a team, but I'm running the project. And we are
going to be using a Kinect sensor in our long-term public art project, which is interesting, and we're
using it for good reasons. But also challenging to think about how do you design for technological
installation that's supposed to last for 20 years? So that's a lot of what I've been doing recently. It's a
memorial for a labor leader in San Francisco names James R. Herman. They're building a new cruise
terminal that they're naming after him. We're designing an interactive experience to tell a little bit more
about his life and then have sort of an installation in memory of him. So there's a touch screen that you
can have -- sort of learn more biographical information about him, and then there would be like LEDs
on the wall animating and displaying different quotes from him. So that's what I'm working on these
days.
Rick Barraza: So Genevieve, just -- you started off the conversation -- I put you on the spot, but you
started off the conversation saying you're not technical, yet everything you just talked about is deeply
technical. You're on the OF DEV Avis for the C++ libraries. You have so much of this stuff. Just
suppose for the fact that almost all of the slides that you're showing in Denmark and ITP, the gender is
pretty balanced on all of your photos. So is there a correlation there? I mean at a certain point, why
did -- where do you find this issue with you, you know, kind of saying does the word technical -- I'm
not a developer, does that come with it, already a certain since the gender that you're trying to separate
from? What's going on there, because you are a deeply technical person?
Genevieve Hoffman: I'm sort of -- so I would never call myself an engineer. And I don't know if I
would necessarily get a job at Microsoft doing anything technical. I'd probably be a designer. But it is
interesting to be in this world of -- because you know, like you get credibility by writing good code. So
I think on a certain level it is pretty democratic. Granted there's been sort of less focus on sort of the
role of female technical education and how that fits into it. But I think in a place like CIID and ITP,
and because maybe the idea of what technical literacy is, it's a little bit more fluid, so I feel fine like
playing in this space. Because usually I'm collaborating. So that's the thing. Like the process of
collaboration and just being able to find the common ground in the language to sort of talk about the
same things I think is what's really important. Because maybe I'm not going to be the fastest developer,
but I still want to have input on what's going to be developed. So I think that's sort of what needs to
happen. And of course I'm a huge proponent of more female engineers. I should have gone to school
for that. But, you know.
Rick Barraza: Thank you very much.
Genevieve Hoffman: Thanks.
Rick Barraza: So two more conversations. As we're seeing this thread, Anna, can you come up really
quick?
So we're seeing a redefinition of who exactly uses code and computation and why. And I think
Genevieve and with ITP, the schools that are coming in and how they're teaching a greater mix.
Now, Anna, you're not on the program, but you are creative; right? So let me actually load up your
slides. And if you talk a little bit of how you've been interfacing with technology on the art side and
how you feel about that blurring of the line.
Anna Czoski: Is this okay?
Rick Barraza: Yes.
Anna Czoski: Can you hear me?
Rick Barraza: Give it a minute.
Anna Czoski: Can you hear me now? Can you hear me now?
Well, I went to school at DX Arts along with James and a few other people here. I'm obsessed and
captivated by motion and different ways you can display it, different ways that you can look at it and
visualize it. This is one example. I'm intrigued with change in movement over time. I took a look at
dancer's motion, specifically her negative space, over the course of -- I extruded it over time. Here you
can see the discreet versus the continuous. I extruded it digitally in my app and then contrast it with the
physical eye's sculpture. And so I absolutely have to use technology in order to realize these things.
This work was made in Unity. It was -- actually I put these slides backwards. This was rendered
Realtime in Unity. I collaborated with a software engineer, Pete Moss, and a dancer, Laura Garcia, to
make this work. That really did blur the lines of video art versus what Unity is usually made for, for
games. And so we used the Kinect to take a look at her motion, and really the technology allowed us to
explore memory, and memory is our core theme. The idea that memory is plastic, that it changes
through time as you revisit a memory, the way we use technology in order to emulate that. This piece
was never the same every time you viewed it. So there are subtle difference that we couldn't use
without it being rendered realtime.
And so this is an interesting project for me because it was more of a cinematic interpretation. In the
past I used a performance and I used a -- I worked with a dancer to extrude motion over time live,
which gave its own esthetic. But here this was a mix between like a poem and a cinematic video art
piece.
Rick Barraza: One quick question. Can you talk a little bit about how your work flow worked with the
creative with the Kinect in Unity and the developer?
Anna Czoski: Sure. For this I was kind of the imaginary up and the instigator of this project. So my
collaborator, Pete Moss, is also an artist, but also the technical guru. We thought of a crazy idea and
then we brought on the dancer. We put three Kinects around to capture her movement. So we got three
angles of her movement. And then put it in Unity. And so then I worked with Pete to create the
breakdown of how this piece would come together, of how the changes in scenes and scene order and
how her movement would change over time. And so we worked very collaboratively to make those
emerging concepts.
Rick Barraza: Are these animates? Are these stills from an animated film? Or are these the stills
themselves, the final product?
Anna Czoski: These are stills from an animated realtime-rendered project.
Rick Barraza: Does any exist on like Vimeo?
Anna Czoski: Yes, you can watch them on Vimeo.
Rick Barraza: Off your name?
Anna Czoski: Off my name, last name is my website and also my Vimeo account.
Rick Barraza: Great. Thank you very much. [applause]
Rick Barraza: The one point on that to call out is we talked a bit about openFrameworks from the
academia and the pedagogue of really make making it accessible to these types of artist coder students
who aren't really focused. We talked about Cinder and its use. This repurpose of Unity has been an
interesting thread that's kind of emerged a lot.
I know, James, you did that work with Unity on Sniff; right? That was a Unity based [indiscernible].
Josh, will you do any work with Unity on the fog side, not too much? So that's another thing we're
exploring and how Kinect V2 will plug in there.
I think our last speaker for today, Michael, can you come up? This aspect of not only how are we
bringing design -- or into the engineering process or into the design process, and this changing face of
developer as more and more industries are leveraging computation. Michael is a computational
architect. I'd like you to introduce yourself and what SOFTlab is, and the emphasis here would be on
how creatives and creative industries are leveraging computation in very new and unique ways.
Michael Szivos: Thanks. So I have a small design studio in New York City called SOFTlab. My
background is I went to school for architecture, so I have undergrad and a grad degree in architecture.
So my mom still asks me why I went to architecture school twice.
We do a lot of installations, special installations. So I'm just going to show one project today, which is
actually one of our first large-scale installations. You know, we do a lot of different other work. We do
a lot of video editing. We also used to do a lot of interactive web work. And now we're getting into
doing interactive installations, which I'm really excited about.
But most of the people in the studio are architects and most of them -- I think most of them more
importantly are tinkerers, so they're not really the best at anything, but they're really good at solving
problems and also not taking no for an answer. So they'll take software and customize it in unique
ways to make these things. So we use a lot of different softwares.
As far as my title goes, if you ask anybody anyone in the studio, they'll say my title is asshole. But,
you know, I basically just kind of move around more now, you know, kind of helping and guiding
everyone. But this project is actually one that I worked on personally. And this was for a group shell
in Williamsburg. So this is a lot of what one train of our work looks like, which is building these sort
of spatial installations that are very efficient.
So it's a structural surface, a minimal surface that holds its own weight, without, say, a structural frame,
so the surface itself is in some ways engineered. And then rationalizing that, so if you had a way to
basically build it, in this case out of many pieces. So we were asked to be part of a large group shell in
a convent in Williamsburg and Brooklyn. And it was kind of interesting, because since it was a
convent it had many small rooms. And we had just got finished doing our first installation, which is
probably a quarter of the size of this, and we're like this is too much work, we're never doing nothing
like that again. And then someone invited us to this show based on the previous show a month later.
And we had a meeting, hey, guys, do you want to do this, and everyone, I was surprised, in the studio,
said yes.
This is part of the research I did while I was in grad school at Columbia where a couple of guys who
also have studios in the city and it was basically figuring out a way to automate the production of a
surface like this. So this is a room we were given. So I'm not going to go too much into the design.
But basically the surface idea was it would connect to all the apertures of the room so that from the
outside you wouldn't see the room itself, you would see the inside of the piece. Which a lot of our
work has to do with exploring these types of spatial geometries.
So to do this we came up with a system where we would produce triangular panels for the surface, and
then each panel would have its own unique clip or connector that basically registers or articulates the
angle between the panels. So the piece I just showed has about, I think, triangulate, it has about 1600
panels. So each panel is unique and the angle between each panel is unique.
So in this case, this is something I think literally -- I mean this is a big argument in architecture whether
things are drawn by hand or done by hand or how they're produced. I don't think this could be done by
hand at all. Even if you were able to find, you know, your worst enemies to do this, there would be a
certain degree of human error where you would measure angles wrong. But you wouldn't want to
invite anyone to this party of drawing this by hand.
So this is research I was doing while I was at Columbia. And it's -- basically we created a custom tool
within Mia. So this uses a [indiscernible], which is Mia-embed language. It was a big deal at the time,
because Mia was one of the few softwares at the time that was kind of very open to customization. So
a lot of big animation houses -- I don't know if you guys can see that very well, but a lot of the big
animation houses use Mia simply because it's a background because they customize it a lot.
So what's happening is this is part of the surface, and it's basically laid out all the triangles flat and just
generated all the clips. So you'll see they're all labeled and they all have the proper angle or degree. So
each one's labeled with adjacent panels and, you know, which panel it goes to and which edge of the
triangle it goes to. So these are the triangles. You can see the notches have also been added to them
and the labels.
So this goes directly to a CNC machine, in this case a laser cutter. So something that just takes these
drawings -- so there's no contractor in between. These are the drawings that go straight from
manufacturing, so the tool path basically. And since we're using Mia, which is an animation software,
basically, which a lot of these parts have history, we can put the piece back together. And this becomes
a drawing we use to assemble the piece.
So this is our earlier work we were doing. And this is, again, as Rick was saying, I mean a lot of our
stuff is Bespoker or zero scale. So we produce tools which take a certain amount of investment. And
some of them are modular so we can use them in other parts and pieces of other projects. But most of
the tools we develop are custom developed for the project. And in some ways it's exciting, but in other
ways it's also tiring. Now we do a lot of work in a cad package called Rhino which has a sort of visual
coder network, I guess a node-based code engine called Grasshopper, which is really nice, because it's
really easy to use.
And so even though we automate a lot of the process, we end up working in the real world, which is a
big part of our practice. So even though we're using stuff that is very precise, as soon as you take stuff
off the flat screen, it becomes very dirty. In this case this is basically a drawing of one of the pieces,
the north side unrolled, and it's covered with black marks because someone's been laser cutting these
pieces for about 24 hours straight and their hands are covered in burnt cardboard. So even though it is
very precise, we still need high tech tools like scissors to customize the things on-site. If some of the
panels are too acute, you can see here, you know, some of these clips are starting to intersect. So this
was carried from our studio at the time. Who thinks it's very funny that now we have to go and sort of
give it a haircut in certain places.
This is a shortened version of this project, but this is a slide I really love to show, because you go from
the precision of a couple of millimeters to three pizza boxes. [laughter] In the sense this is one of our
first installations, as I said. So this is the only piece of equipment that we had really was a ladder. We
didn't bring -- now we come with like drills and things like that. But as people who normally worked
mostly doing video editing and interactive web work in the beginning, which again you just need a
computer. So we showed up with all these panels. Everything's going to work, it's very precise, it was
generated on a computer. We didn't have any idea of gravity, weather, whether it's humid in the space.
And so everyone -- there are forty artists scavaging around. There's one ladder that you kind of have to
share with forty artists. The furniture that's there is stuff we found in the convent, because we didn't
bring in a ladder. And luckily we ordered pizza that day which we used to shimmy this piece, which I
guess we thought was going to float in air until we installed it. And then this is the final piece installed.
And this is sort of the chipboard panels. And then also the final piece was clad with reflective mylar.
Again, this is something we added in the original design because we produce these shapes sometimes
which are very pure engineered forms, but that's not so much what's important to us. But what's
important to us is people discovering or engaging with those projects. So the idea with the mylar
panels was that it would also confuse people the same as it would from the outside that reflects the
surrounding room. You can see the floorboard in the piece, so that people are sort of forced to walk
around the piece. We're very much about producing an experience rather than a sort of objectified art
piece or something that's on a pedestal that you look at from far away, but more something that you
engage with, which is something that is very exciting for me with some of the other more interactive
physical installations we're starting to do. But again this is just one project that kind of shows our work
flow. I think it shows it very well in that we're using animation package and we're kind of customizing
it to do what we want. I mean Mia's not a typically software you would use to produce drawings for
architecture.
And that's it. Thanks. [applause]
Rick Barraza: You can sit down. I think we have time for a couple of questions. If you spoke,
Genevieve, can you come up? Anna? James, can you come up? And Stephen? Just take a chair. We
have a couple microphones; right?
Does anybody have any questions on anything you've just seen or anything you haven't seen?
>>: You were saying that it's hard to -- there's no role name for it. And I see there's ad agencies doing
it, there's architecture. It feels like it's happening everywhere. If somebody wants to work doing just
that and not just as a side project, where do they start? Like what title do they look for? What kind of
companies? Is it ad agencies? Is it a piece of Microsoft? Is it a big company? Is it a small -- you
know, it's just hard to know where those jobs are.
>>: The term that is getting thrown around the most is creative technology. And I think it's starting to
find its home a bit more these days. For the past couple of years, creative technology has been used to
define anything that involved programming or design. But I think we're starting to see creative
technology defines the act of using technology as an artistic medium. So if you see -- if you're looking
for a job in that, you know, you will see the term creative technologist, and upon investigation you'll
probably find out if -- what the roles of what the company calls a creative technologists are. They
might just be programming, they might be doing what it's supposed to do, which is to use technology as
a medium of artistic expression.
Rick Barraza: Andrew Bowden couldn't be here, but from Cinder, they've been exploring the term in
film, they call it the technical producer. That's kind of a very niche industry. And I think from the
architecture side, it's very interesting. Because we've had an ongoing conversation for a while now that
in software we kind of let that title get misappropriate, either you're a software architect, and actually
more a structural engineer. And architect, despite every other industry, is someone who's -- they don't
build buildings, they're creating experience, like you were saying.
How do you feel when you hear software architect? Do you think in software it's kind of misapplied?
Is the word architecture, is that what we're looking for? Does that create this balance between
engineering and creativity? What's your thoughts on that?
Michael Szivos: Well, when someone just says architect, I'm sometimes confused, what does that
mean, are you software architect?
Rick Barraza: Or a real architect?
Michael Szivos: I think an architect in, let's say, the larger context of building, is more like a director
in a film in that maybe they're generalists, but they manage maybe different professions on a bigger
project. But they're sort of design background, but in architect office. And that's the sort of big
struggle. There's always project managers and designers and lead architects and project architects,
same as in an agency. And I think when I first -- before we first started, we were doing a lot of work
that was more videoing. And I was put on sometimes animation teams and because I knew MEL, I was
a technical director basically, which is what it was called in the animation studio.
And I think that's what you're saying earlier, you're just solving problems but you're not given any
design direction. So I believe the ability for a creative person to customize the tools that they're using,
I mean whether it's a painter or not, but imagine having a brush that you could make it any shape that
you like. I think that's what's happening now. I mean this stuff is very big in architect right now.
So there's a lot of people as in -- I don't know what you call it, building architecture, that are interested
in code and computation and generative techniques for making, let's say, shapes or form. But, yeah, I
think it's the difference between someone could drive a car but can't really work on it. And then there's
the person that can turn the car into a hotrod, and then there's the person that can build a car from
scratch, which is very rare. But I think now there's a lot of tools that allow the in-between zone where
someone can customize a hotrod car. And that's what we tend to do in the studio. None of the things
we're making we're building from scratch, but that's -Rick Barraza: It's that maker-tinker spirit you're talking about. And Genevieve, the question was on
getting a job, but I think before that it could actually be one on education.
And I know Lia, although you're sitting down here, you could be sitting up here too. You're doing a bit
on education with very young people using processing; right?
And Genevieve, in terms of what you were teaching at Denmark and ITP, how do you think education
is changing to embrace this kind of blurry line of artist-computational-tinker-engineer slash developer
creatives?
Genevieve Hoffman: Yeah, I think that's a lot of the work that processing does in openFrameworks
does, just to get out-of-the-box coding environments up and running for people really quickly. I think
that's a huge barrier that has been overcome in education. It's a way that to get immediate visual
feedback, which I think is important for -- some people, designers, kind of going towards coding.
Yeah, I --
Rick Barraza: It seems from a lot of us from the software side, our first project is Hello World. Yet for
a lot of -- for everybody else it's a red circle. That's kind of that difference with your first project, a red
circle moving across the screen, or Hello World, forces that early split if you're going to be a traditional
CS'er.
James, he's not here, but can you talk a little bit from the school of computation of what that's trying to
happen from the educational side?
James George: Yeah, I mean, I think that in response to your question directly, it generally doesn't
come with the term first. In terms, self association comes afterwards where you're like, oh, I'm doing
this thing and now I have to describe it to somebody. And that's when it becomes problematic. But it
tends to be that you're already doing the work at that point. So it's like a lesser problem.
So the idea of creating a term or a name for yourself, like a word for it, and starting at the same time, I
think is a little out of order. So your point, so Jack Lieberman, along with Jen Lowe and actually Tyde
and Troy, a lot of people in this community are starting a new school, which is an alternative
educational model called The School For Product Computation. I'm not involved with it at all, but it's
sort of part of our community. And they have a new set of terms also that have this idea of
computational poet. So they don't associate with any of the artists, because there's a lot of baggage
with that word, a lot of ego associated often, and rather this idea of creating something that's poetic.
Because it has sort of an intrinsic cultural or emotional value using computers for computation.
Rick Barraza: So on the far -- so back to Stephen, I guess for the final part of this answer. Because
you're the most -- you're an agency that works at this and you hire people. How in the world do you
look to hire people into The LODGE if you do hire externally?
Stephen Scieberl: We say that we hire wrong. We -- yeah, it's -- it really is on a case-by-case basis.
People -- fortunately we're in a position where people seek us out. There are a lot of people we go and
look for too. But there's -- I don't think you can really say that's one thing that we're ever looking for.
Even when you look at our department as a whole, we basically have a lot of people that make up this
really nice mosaic that just results in this really great things happening. But we kind of run the gamut
of people who are really -- who fall heavier on this sort of visual and user experience side and then all
the way over to the full-on engineer that can do things on a really low level. And it's really -- I guess
when you're trying to put a department together, like what we have, it's less about just trying to fill
specific roles and looking at what an individual does and where they fit into your bigger picture.
Rick Barraza: Do you think the company culture has as much of an effect? I mean can this role exist
in a company that doesn't optimize for design or optimize for creativity?
Stephen Scieberl: It's tough to say. What we -- we have -- the ethos at Weiden+Kennedy is probably
not quite the same as somebody that you would find like at a publicly-owned company where there's
shareholders to appease. We pride ourselves on taking risks. And so I wouldn't say we have all the
answers, but we, you know we put ourselves out there and we take risks in hiring and what we do. And
it all seems to -- for the most part it seems to pan out pretty well.
Rick Barraza: Any other questions on any other topics? Oh, let me get you a mic so they can hear it on
the video.
>>: I'm just kind of curious, I really much enjoyed your presentations. I'm just curious, what are your
thoughts on the dimension of time and the timelessness of the artifacts that you guys create? My wife
is an architect and we talk about this quite a lot. It tends to -- it's fascinating to me that any conference
room table, any artifact that we live in will probably outlast the technology that we live in, in an order
of magnitude in a bigger way. So it's always interesting to how do these technology-driven projects
last? How do they age and are they really dujour? Are they like right now in the moment? And the
whole point is they're just a drop right now and that's what it is? Or is there some thought given in the
design process about how do you account for the fact that Kinect is interesting right now, but is it really
going to be interesting in like five years from now and what's interesting about it? Is it the fact that all
of the sudden the depth sensing became easy and cheap to do? Or is it -- and is that going to look kind
of naive ten years from now looking back at what's happening right now? So I would just like to hear
some thoughts on that.
>>: I have to think a lot about that right now because I'm trying to make a 20-year maintenance plan
for a technology installation. And I think, you know, it's a toss up, because I want the immediacy of an
experience that responds to you. I would like it to age well. I'm going to provide guidelines of how to
do that and buying backups of pretty much every piece of equipment that we'll be using to replace them
over 20 years. I've worked in museums also in conservation departments for media art works that have
been collected by MOMA and Whitney. And it is something that tons of people are really trying to
figure out right now as artwork kinds of moves in this direction, as our lives move in this direction,
because we sort of design for obsolescence or just having the faster thing sort of available within three
years. But if you designed a piece of art to function in a certain way, you can't just replace that
computer when it breaks down. So it's either a matter of using old pieces of technology and trying to
keep them updated, or literally sort of a museum collects a piece and you give them a circuit board
diagram, because they're going to have to prefab that at some point. So there's various ways to sort of
figure it out. But it's true; like this medium doesn't lend itself to longevity necessarily.
>>: Yeah, essentially with advertising, a lot of times things that we're making, it's a very ephemeral
nature to it, because we're making something that's supposed to live for three weeks for a campaign.
Sometimes we're going to make something that will last forever like our "the work comes first" wall.
That's something that we want to be there for a long time. But if you look at it, we started off with old
technology, electricity and so on, and a couple depth cameras. And it should be able to -- and then we
built that to last for a long time with a plan in place to keep it alive as an art piece. But the commercial
role I think is a little different than what a lot of these guys are working on and that we're pushing
technology really hard to do something that is only built to last for the duration of the campaign.
Rick Barraza: James, this is one of the themes in your Clouds film itself, right? Because you didn't
approach it as a tech demo. It's the difference between a prototype and a vision type. You want to tell a
story about technology and you're not saying, look at me I'm using the Kinect to do depth data. It
wasn't about the tech, it was about the use of it. But from those interviews, because this is a very
recurring question, watch Clouds. In Clouds this is a common thing. What are your views on this?
>>: Yeah, I think that we struggle with that a lot in the project, and especially in this idea that making
design fiction and if history goes in a certain way that it can render your work irrelevant or can make it
seem, if it's still -- if it actually happened, then it just seems banal, because that's the reality we live in;
it doesn't seem that provocative any more.
Rick Barraza: Nothing inspires faster than novelty.
>>: Sure, I think that one thing I'm attempting to do with this work is one thing that never will --
history will never expire, in a way, because you create a portrait of what was happening right now, and
if it's told as a human story, it's something that's part of a lineage or a genuine community that exists
that, sure, they're galvanizing around technology, but it's really about the interaction they're having, the
structure of the interaction, this community of artists, ourselves included, then that becomes hopefully a
more timeless portrait because it was told using the medium of the time that it was happening. So that
was the choice there.
So it's not about -- it's not anchored on the novelty but rather it's anchored on the reason that it looks
that way or that it can be told using that tool. So then looking back it may feel old, but also the idea is
that they're talking about -- will be grounding in what was interesting to us, you know, in this period.
So it would become like an archive.
Rick Barraza: I think from the business side of things, it's fascinating, because when we're talking
about zero scale,w that attitude is about exploration, creative, the freedom to fail, following your
passion. And all of you, all of you, part of the [indiscernible], your jobs are kind of making the
invisible visible for the first time, really pushing those boundaries, see what sticks. And the first step is
modifying that and selling it to high brands who are looking for the next cool thing to really -- not just
that, but understand that agency space.
But then you start bringing in components eventually it scales out like with your Cinder plug in. When
you were doing it -- when Kinect just came out and you were hacking it and there weren't plug-ins or
anything for it, it was the first time, but it triggered people's imaginations. Two years later, it's part of
the component that comes in the SDK for everybody to use it at scale. So it's not that there's zero scale
in a chasm; zero scale sets the direction and tone for these explorations that, best case scenario, does
become the knob, because five years later everybody's doing it. But that's actually your victory
condition, right? I mean, you changed technology.
>>: Right, and that's something that we're trying to do with our brands as well. Like with the work
that we're doing with Taylor's X for example where right now we're working on changing the way that
you do [indiscernible] and there's going to be an execution around that. The execution will one day be
superseded and that will go away. But the bar is going to be set. Like your whole approach and
understanding of taxes is going to change and it's going to be made easier. And that's going to be the
place where things are going to start from in the future. So in that sense that is creating lasting effect.
Rick Barraza: I think we're almost out of time. Is there one more quick question? Let me grab the
mic.
>>: I'm curious on the tool side of things. So I started out as a musician. I'm marketing here, not
technical at all, hated math growing up, but started getting into coding a couple years ago because I was
fascinated by the technology and what could be done. And it seems like the ability or the maturity of
the tools has the capacity to unlock new ways to, you know, express ideas and be inventive. So I'm
curious to know where you guys think the -- from a tool perspective, like what are the things that are
really interesting? Where do you think things are going? Like Brett Victor talks a lot about -- had
some really compelling ideas about how the tools should be evolving, and I'm just curious to get your
takes on that.
>>: Well, I guess maybe define better what you mean by tool.
>>: So -- well, so, it seems like there's -- processing seems to be one that stands out, gets mentioned a
lot. Apple just yesterday with Swift, how they've got this more sort of interactive environment in
which to code. So that's ->>: Yeah, I think that -- well, I think about this a lot contributing to OF and how watching how
openFrameworks, which is in a lot of ways similar to processing, has changed. In a lot of ways that it
passes itself as being progressive, like it's always enhancing, but actually in a lot of ways it's just kind
of a moving target and it tends to follow the trends of what people are making. So in a way I can say
they are maturing. But I think that that, if I were to be truthful, isn't necessarily a -- like an accurate
description of what's happening. But actually just adapting to different fads and interests. So if you're
looking like from an art historical perspective, you think of tools as being what supports the ideas that
people want to talk about right then, you can see trends in there in the fact that a lot of stuff has moved
from 2D to 3D and things evolved a lot more about sensing more than just pure visualization. But then
in that process it's actually lost a lot of ways that we used to work with, even in the last ten years, in
terms of pixel manipulation or different types of line drawing and specifics there. So and you can trace
this through the projects that people are making. So like processing and things like openFrameworks,
which may be different than ideas in Unity which have much more of a road map in front of them, you
end up looking at the artwork that people are making or the design projects that people are making as
the sort of canaries in the coal mine for where the tool will move.
Rick Barraza: How do you think Open Source has changed the conversation around tools? Because
we have openFrameworks processing Cinder, they're all Open Source. I mean ->>: Yeah, I think it's -- Cinder's a little bit different of a story because we have sort of two levels. We
have the core, which especially right now is we're making sure that it is becoming something that is this
really, really solid professional toolkit on which you can rely. But at the same time is Open Source and
has a community around it that allows you to plug things in and make your own manipulations to it. So
it sits in this kind of interesting space of we want to make a really professional product that you can
completely hack. So as far as maturity goes, I have to go with James the same way. There's a lot of
things that are changing. I think the core is maturing and it's becoming something we can rely on more.
I think that's kind of happening with all of the frameworks is as people play with them and started to
use them in ways that they need to make a living -Genevieve Hoffman: I think it's also not just one. Like I was trying to think what's my tool? And it
kind of depends like what I'm trying to do. So what I definitely developed at ITP where I got a kind of
more structured programming education, I just want to know enough so that I can get my foot like
learning the next thing that I need to learn. So a lot of what I'll be doing is like using the right sort of
node base framework if I want to do some socket stuff, and then it will connect back to something else
through SC. So it's just like a lot of patching together the right tool for the right application and then
putting them altogether somehow. Is a lot of what -- and I like first use Max MSP, so I was connecting
little boxes and I had like a little programming block that I could kind of put together. So that was like
a nice introduction of, oh, this is sort of the way to think programmatically. But I think the nice thing
about openFrameworks and Cinder and processing is you're not fed those little nuggets. It's up to you
to kind of figure out what you're going to create. So it's a much more blank canvas approach, I guess.
>>: And I think the main thing you're starting to see lately as more and more people are getting
involved is these tools are working better. So what it comes down to, I think even five years ago -- you
know, there's still going to be new libraries that have missing pieces, but I think five years ago you
would get a library or framework and be like, oh, I want to go do this thing, and you would find in the
code there's like a section missing, you should write this code, here. And you starting to see less of
that. It feels more and more like I need to do this thing that's kind of out there and maybe this one
particular technology doesn't exist, but I can grab these two libraries that already work great that a lot
of people are using and put my own thing together. And more and more it seems like when you reach
out to the Open Source world, you're getting something you can rely on.
Rick Barraza: I would think -- you mention tinker. And I think that's one of the common threads here
is not all tools are equal and some tools are very tinker friendly.
I mean processing, Lia, you're teaching processing to very young children here when you're not
working on the Internet Of Things, all the way up to Jared Thorp being on stage at Ted using the exact
same processing, giving talks for New York Times and data visualization; right. Because it has
intelligent defaults. And I think at Microsoft many of our tools turn into components sometimes. And
it's like you're young, learn scratch, you know, maybe JavaScript, then VB, then C#, you got some gray
in your beard, C++, right. Because those aren't necessarily tinker languages out of the box; right? I
mean in openFramework, C++ library, one line of code, I've created a texture, image, load this file in
my draw loop, one line of code, draw the image; right? If I wanted to do that with C++ and Direct X
Flat, it's like a thousand lines of code because I have to understand back buffers and setting up
parameters before I can do the swap chain and then pushing it out. That's not tinker friendly. But OF
abstracts C++, makes it tinker friendly, and we have artists coming in and creating amazing things that
traditional engineers, it's just -- it's in our blind spot.
So I actually think the tools question is a very, very important question for Microsoft to dig into, simply
because we're so good at Legos, and yet these abstraction layers let's you play with them like clay, like
let's you tinker. So I actually think it's a critical point for Microsoft to be hearing exactly those
questions. What we're going to do with the Internet Of Things, will it be tinker friendly? Or will it
already be optimized for professionals?
Thank you very much. We went all the way up to 3:00. We're going to be here for -- actually thank
you everybody [applause].