>> Desney Tan: It is my pleasure to introduce Golan Levin, who is a professor out at CMU in computation art, but also in design computer science and entertainment technology, because of how broadly his work applies to art, science, technology, culture, so on and so forth, all things we're trying to do here. Golan is an impressive guy, renaissance man. Probably more impressively seems to create renaissance teams wherever he goes. He currently runs a studio for Creative Inquiry out at CMU, but also been at the MIT media lab, interval research, Electronica, the list just goes on and on. Amazingly impressive. Probably the last thing I'm going to say because I'm standing between you and a wonderful talk was a citation that I found online from a recent award he won by Fast Company. He was named one of 50 designers shaping the future. And the citation read: If CMU is to technology as Hogwarts is to magic, Golan Levin is Dumbledor. And it was just amazing. >> Golan Levin: too kind. Thank you. Thank you so much, Desney. That's really I am really glad to be here. You all are one bunch of really smart Fer's. So I'm a little daunted. Also, when one is preparing talks like this, you never know who you're going to encounter. So I had heard that Microsoft engineering was real engineering heavy. Microsoft Research was like super engineering heavy. I just came from a presentation I gave to a bunch of engineers, none of whom at CMU, none of whom had heard of Samuel Becket. And it was like -- I was like so how many of you know about Samuel Becket. No one raises their hand. It was like: Oh man. And it was important to my talk. I had to kind of do a bunch of backtracking. So it's always hard for me to initially gauge the audience. Today's talk also I have to say I have to warn you in advance it's a little bit of a mess. I've actually been preparing it for a while. But the reason it's a mess is because about half of it is completely new customized just for you guys. And it has to do, the first half is basically a talk about me and my work and sort of my pedagogy and the laboratory I directed, the projects we're doing there, my own artworks and things like that across new media. The second half of the talk comes from a discussion I had with Desney, because I understand from Desney that y'all are thinking about how to integrate arts-oriented thinking into a research lab environment. And it's something I've been thinking about for a long time. Just as an artist myself, I have about more than 20 years of experience working as an artist in high tech research laboratory environments, and I also know a lot of people who also have and have also been thinking about how to do this. In fact, for example -- and I brought this copy for you guys -- my laboratory recently put out a book about the history of like artists in research labs. So it's the kind of thing, like it's on my mind, it's on the mind of my friends, and there's a lot of like sort of historical screw ups and there's also like a lot of like awesome amazing things that have happened as well. So with the opportunity to talk to you about arts thinking or artists in research laboratory environments, I thought it would be kind of fun to kind of show some new thoughts. But they're kind of still kind of disorganized. And one can dig back into the history of this kind of thing to a great deal of detail. And I haven't found all the cool videos from the 1970s that I'd like to show you. But you can imagine them. You're probably Googling them or Binging them in real time. So I'd like to begin with this just because this kind of introduces some of my work in a way that talks about a couple of my, two of my abiding sort of passions and curiosities and interests. Anyone know what you're looking at here? Blob. Implicit surface. Good. Yes, you are looking at an open GL rendered surface, but what else? Anyone got -- yeah. >>: Should I say it? >> Golan Levin: >>: Yeah. Do you know? >> Golan Levin: This is my friend James. James, who is that guy. James is going to be talking tomorrow. And he'll be showing his work in Kinect-based cinema. So we're here together at Desney's invitation and also of XBox and entertainment. So I'll talk about his work. could be? But anyone else got a clue what this >>: Short test. >> Golan Levin: A little bit, actually. Obviously it's a bit mysterious. These are -- hey, I'm so glad you made it. These are hands. But they're a particular set of hands that really means a lot to me. They're the hands of this guy. I'm just going to switch around and do the turn off mirroring and actually launch into the presentation. Play. The hands of this guy. This is Merce Cunningham, shortly before he died consented himself to be motion captured by some friends of mine, the open ended group, Mark Downey and some other folks in the media lab, who have a small crew of arts researchers. They were basically rigging Merce with motion captures. Merce as you know basically invented, along with Martha Graham, 20th century modern dance. There was ballet through 100s of years and Merce and Martha basically said we can invent a modern language of dance. And by the time this was recorded, Merce was of such advanced age that he could no longer dance with his full body. But he could still dance with the parts of himself that he could dance with which were his hands. So in this case you can even see the little retro reflective capture spheres on his hand, Viacon system recording him. And he consented himself to be recorded this way. Shortly afterwards he passed away. Open Ended Group invited people to interpret this data. And for me this was a really great opportunity to think about two abiding interests of mine. The first being gesture, because I believe the way we moved through space is something that really helps define our spirits and helps articulate our unique identities in the world and space and time. And also visualization, which is a really interesting problem that I think also you guys think about a lot is how to take stuff that's kind of ineffable and make it visible. For me this is kind of a poetic exercise, too, giving back to Merce Cunningham a body he lost. That was the idea of taking these particles in some sense and using them to scaffold a new kind of virtual meat that would become his new body after he had passed away. So that's -- I show it to you because it's kind of a quick introduction to gesture, mark making and visualization, which are a couple of things I really care about. So today my talk is called new media arts inside and outside the laboratory. Really honored to be here to present this to you guys. And it's a talk in two parts. First, as new media arts themselves are manifest by instance in my own work, pedagogy I'll be talking about, my organizational activities, and secondly some new thoughts as I mentioned on arts and research environments. So who is this guy? A few of you know me. I'm delighted to see some familiar faces here. I'm an artist. For the past 23 years I've worked as an artist in the context of various research laboratories, including seven years at the media lab at MIT. Five years at Paul Alan's former Interval Research. How many have heard of Interval? That's awesome. Not many people have. It's an interesting little footnote now. And I'll show some of my own work shortly but in a nutshell I wanted to say that I feel that artists, sort of not necessarily proscriptive for everyone, but it's proscriptive for me. I feel artists obliged to use the medium of our time, which in my opinion means software and circuitry as an artistic medium. And to be clear, I write my own software generally C++ or Java, and I'm interested in exploring the new forms of aesthetic experience which are made possible by technology. And also the cultural implications of new technologies and technology itself. And I generally do this by either inventing or adapting technological systems. So how many of you are familiar or have heard of the phrase new media art? Okay. So that's the -- I don't prefer categories, but when I get categories that's where I'm generally called and that's the work I do. I'm an educator, since 2004 I've been at Carnegie Mellon where I've worked to create bridges across technology and engineering. My home appointment is in the School of Art, but as Desney mentioned I have courtesy appointments in design, computer science, entertainment technology. I'll be showing some of my students' work later in this talk including some that was supported by Microsoft Research. I'm also a lab director. I direct a small research laboratory called the Studio for Creative Inquiry. It's been around for 23 years. I took it over in 2009. Its mission is to support atypical anti-disciplinary and interinstitutional research projects at the intersection of arts, science technology and culture. We support a wide variety of new media research which I'll show today including some of which that you'll also see tomorrow in James's presentation, a forward pointer to James who is a fellow in our lab, who has been working on Kinect-based cinema. And finally kind of a communist. In the background of everything I'm going to be showing you today is my participation in several different communities for commons-based peer production. I hope that's a phrase you guys recognize, but if not it's basically hackers out in the world basically grouping together and working across sometimes very large geographic boundaries on collective projects involving typically open source code. If you've heard of, for example, Arduino, it's an example of such a collective. I don't happen to be involved in that community but I'm involved in several others like processing and open frameworks that I'll be talking about. These are widely distributed collectives of makers who are developing free open source tools for arts engineering. These are programming environments made for artists by artists that have enabled tens of thousands of creative people to become software creators. So working in the arts and education resources are often very scarce. Much of my work and that of my students, because we're in the arts and we don't have a lot of money, depends on the free and open source tools and technologies. I try to give back to these communities as well by posting code, by organizing activities that bring the community together and support that community. So that's a quick overview. A practice in new media arts. So me now and some of what I do. Some core concerns are the aesthetics of interaction of data. I'm interested in basically creating provocative examples, in particular new kinds of interactions that we can have with data of all kinds, whether it's data from online or data that we generate ourselves. I'm interested also in this idea of expanding our vocabulary of action, by literally allowing people to discover their own potentials creative agents in new ways, usually that involve play. So how can people discover themselves in new ways and how can people become, discover new voices in themselves for acting in the world. And those are the kinds of concerns I have. I have some ways I like to work. Most of the money in the world is down here in the useful section. And that's fine. That's good. I tend to work over here. I work there. It means I'm poor. But I work there because actually so much attention is put on this sort of part of the space that actually there's a sort of huge undiscovered territory over here that eventually perhaps migrates leftward and downward, but there's like sort of large untrammeled territory here and lots of unanswered questions, because very few people can afford to not look there or can afford to look there, I guess. They're all focused on the lower left. So that's one place where I tend to locate myself. And actually as I'll kind of emphasize again in this talk, and maybe this is since I had forgotten to add the slides, this is now the time to consult my handwritten notes and just mention that there are sort of two trends in new media arts that you'll see both in my work and that of others, that of sort of the progressive more visionary and sometimes exploratory and inventive and that of more the critical, the mirror of society. Sometimes you can kind of say that's somewhere along here. Is it more about aesthetics, more about kind of creating quality of life and creating kind of beautiful experiences or sort of sumptuousness or this kind of visionary thing that transports you, or is it more about something that makes you think about the world and yourself in a new way? So I just kind of wanted to outline that. Because I think particularly when we involve artists and research labs, the way that doesn't shake any boats is to go strictly up here. Right? But a lot of the real meat of society happens down here. And so it's important to hit both. >>: Could you say a little something about what interesting means, because normally I think of that as an insult, like the worst thing you could say about some art is that it's interesting. Because it's just so bland. >> Golan Levin: Oh, okay. All three of these. I think provocative. All three of these are bland terms anyway. Depending on your particular bent, beautiful, it's like oh boy. Beautiful. It's like actually I have friends for whom beautiful is an insult. Seriously. That's like -- it's like oh great, you're saying it's meaningless. It's like I have some friends -- I have one friend Marius Watts, incredible artist, designer. Makes really beautiful objects. He hates it when someone says it's beautiful because not another meaningless object. Thanks. He's sort of finally come to love the beauty and he's like, okay, look, you know what it's beautiful and I'm proud. But he's sort of like made peace with the fact he's not doing this like difficult tactical activist confrontational media, speculative stuff that really provokes. But you need both. And I will actually talk a bit more about this, bookmark for later. The second thing that I couldn't kludge into the talk at the very last minute was Julian Oliver's critical Engineering Manifesto, which I wanted to talk about but couldn't -- it was like: Oh my God it's going to kill them if I add that in, too. So if we get time, I'll come back to this. But I think in some ways all these terms are a little bit disparaging, a little bland. It's actually when we mix them we get useful, get productive and valuable value. Let's say we get value by mixing these things. Also in terms of my own personal mode of practice. So I write my own software. I write it in C++ and Java, and as I mentioned, and to me it's a certain pride in sort of DIY. I don't mean DIY like me and my garage, although I sometimes do. But I actually mean DIY in the sense I take a certain pride in being an artist who is able to use code. And that's just my medium. If that's my medium, I should be good at my craft. So to a certain extent I believe -- and I've been trying to inculcate in my students a certain kind of arts and engineering hybridity. They have to be culturally savvy and they have to have good craft and a good eye but have to make the stuff themselves. And it's not any good if they're dependent on others, please Mr. Engineer or Mrs. Engineer would you make this for me. No, they have to be able to DIY. On the other hand, as you guys know from working in teams, and it's any kind of substantial project in the world requires, you have to do it with others. And so a lot of -- frankly, it's more fun. So a lot of projects involve not working with someone else who is my technical proxy but, rather, working with my peers who have other kinds of skills and voices that they bring to the table. This has to do more with my situation within academia, because it's one of these questions like how do academics get evaluated. But this is kind of valuable here, because to talk about because this is basically making papers. And you know that contain reproducible techniques and algorithms, pseudo code and other kinds of things. This is about making sublime examples that go out in the world and change how people think and change what people think is possible. I tend to work around this side of the spectrum, although occasionally I write a paper where I contribute on a paper. And it's just about having a mode of practice. Like can I create influential examples that sort of awaken people to possibilities of technology and culture that they were not previously aware of. And finally another way I like to work is at the intersection of open source and pop culture. I work with a posse called the Free Art and Technology Lab. And what this group tries to do is to create open source technologies that allow people to sort of mash up culture for themselves. So basically these are, we release tools to the public that sometimes have devastating effects. No. That allow people to sort of take media back for themselves and adapt the media around them. So we release the tools and then it's because we're releasing tools that people care about because they get to insert themselves and inject themselves into and around media that is meaningful to them. Okay. So some examples. One way that this kind of liberation can happen is by allowing people to discover their potentials as creative actors. I do it through variety of different gesture and audio play systems and interactive surfaces. This just one example from 2004 -whoops. This is an interactive surface I developed in 2004. It was just when the sort of surface rear projection, what you might expect series of flocking animals that form a poetic experience for bar tables, rear projection. This is 2004. This is not too much of a surprise for you guys who are producing the surface, the first surface. But, I mean, so it's just basically how can we sort of enhance quality of life in terms of a social experience with this kind of augmentation. Another example is this here, which is a table which presents the fiction that the table is a musical score. And any objects placed thereupon become elements or musical notes in the score. So it's got an infrared camera, tracking the surface, what I call a simple projected augmented reality, provides an additional layer of information. And time goes one direction, pitch goes another. And this kind of shows that. So it's doing basically kind of a Fourier synthesis. Gating a bunch of oscillators in relationship to the objects it sees there. And all of this is sort of virtual projection, indicates octaves and divisions of time. If there's a vertical mark, then it sort of opens up all the oscillators. Get kind of a percussive sound. And it's possible to make some funky beats and a kind of performance environment. Sort of like VJing in a way. You're not jockeying other people's videos, but you're jockeying this kind of visual situation. Another abiding interest of mine has been visualizing communication and also visualizing the voice. I'm delighted to see Andy Wilson here because this project here, which I'm going to have to show, first of all, it's one of my first projects in 2001 as a professional artist. But it was concerned with this idea of people's telephones as pixels. And I noticed you had a paper on that. And what's happening here, this is going to be dark ->>: [inaudible]. >> Golan Levin: Okay. This is a project, really one of my first projects as a professional artist, which was a concert performed entirely on the audience's mobile phones. Basically people would register their mobile phones beforehand outside the venue, about 200 people in the audience. They would give us their mobile phone number. And we would generate for them a ticket telling them where to sit. We would send them a new ring tone to their phone, and then because we knew their phone number, where they were sitting, what sound they would make, we would make waves of sound that would cascade across the audience and make cords dialing people simultaneously and melodies that would snake throughout the crowd. And each person's phone would also light up in this kind of atmosphere become a throng pixel. [music] This is our interface for launching the phone calls. [music] So we were able to dial up to 60 people simultaneously out of the 200 who were registered into the system. And I forgot to mention there's a huge projection pointing down at them. So as you would get your phone call ring, your phone would light up. So basically a big spot of light from an enormous projector overhead. Each person was kind of an audiovisual pixel here. The purpose was to visualize this kind of information sphere that we're constantly within and don't necessarily always palpably feel. So we're up on stage launching these phone calls into the crowd with this interface. Okay. Thank you. Perfect. Another project which was from 2003 with Zach Lieberman and John LaBarber and Yop Banck [phonetic], a project. Concerned with visualizing the voice. In this project here it's trying to present the fiction that speech cast visible shadows, takes the shouts, the barks, speech and song of two experimental vocalists who are basically people who make funny sounds with their mouths for a living and attempts to visualize all of their activities in this kind of extended cartoon space. So here, for example, is Yonk Banck, he's an experimental Dutch sound poet. He's making what he calls a cheek synthesizer. Sounds a bit like this. Pppffft. Every single cheek flap there turns into a ball. So kind of rapid stream of balls that are kind of coming out of his mouth as he does this. And here's the basic scenario. You see it's a very large display. It's in this case a three-screen stereographic projection. Computer vision system that's identifying where his head is. There's a wireless mic being analyzed by a sound analysis system and obviously a big computer graphic simulation that tries to project graphics that are expressively coupled to speech appear to coincide with the apparent point of origin. This is one scene in the concert. The concert is 40 minutes long, 15 different scenes that go through a series of changes and tell an arc or story about these two characters. Here's John La Barber, you can make out her body underneath this voice sensitive costume. We had about 15 metaphors by which we could interpret this world. In this case here it's pegged to her body via computer vision system. It's a voice sensitive system that listens to her voice and modulates its periphery as a result. Here's a voice sensitive environment. A voice generated environment. This is kind of another vocal effluvium, this fluid, stem that comes out of their mouths. Here's one where they're painting with the voice. Just to give you a quick view of what these look like I'll show two scenes, this one here. Now, what I haven't mentioned is that every time he makes a little sound, it stores that sound in the corresponding ball. When those balls fall down and hit the bottom edge of the screen they're going to release those sounds. Creating kind of a chaos. Okay. Now, in another scene, this one's kind of a like a David Attenboro nature special and Yonk is speaking in abstract Dutch with kind of abstract subtitles. [Noise] Okay. So as you can see, this is not rocket science, but there's a great deal of poetics in terms of the considerations happening here, both in terms of storytelling, if you will, and also in terms of the visual experience. After making this, we realized that we were actually able to return these 15 or so different scenes to the public in interactive form. Although some of the scenes were really highly tuned for these specific vocalists and their particular virtuosic techniques. About half of the scenes can be returned to the public and used in interactive situations so people wouldn't have to experience it vicariously but could experience it firsthand. These were sort of participatory scenarios, these are the children like three and four-year-old girls in France. And one more time with the lights. [Noise]. For me, this is really -- for me, this is actually one of the most important moments in my talk because what's really happening here he's more than just kind of having fun with the system. Or discovering what a system can do. He's actually becoming someone or something else. He's discovering kind of a new identity himself. In the same way that one can be transformed as an actor by wearing a mask or a costume. He's really discovering a voice in himself that he wouldn't have otherwise. And this ability to discover as I said his own potential as an actor discover this new voice in himself is a crucial moment for me that is enabled by interactive media. [Singing] He's five. All 5-year-olds learn to sing this song in France. And then they forget. These are teenagers, they're kind of singing random notes. The pitch is governed the way the curbs bend, making rising tones, curve clockwise, making phone tones curved counterclockwise. I'd like to show another project I did with Yop. Actually leave the lights low because it's going to be a little bit dark. This project here is another one concerned with speech visualization. For me it was a unique opportunity to do something we don't often get the luxury of doing. Maybe it's an argument for considering sort of high-end content in a research environment. Here's why. It turns out that Yop is the world expert in performing this very special poem called the Ersa Nada by Kirtsch Vitters [phonetic]. The Ersa Nada is a master work of 20th century abstract concrete poetry. Half hour long poem of purely strictly patterned nonsense. Took Kirtsch ten years to write. And probably about half a dozen people in the world that have the thing memorized, and he's one of them. So he's able to deliver this really insane nonsense poem that sounds like just gibberish, very, very fast, and very, very accurately, and that presents a unique situation if you have something like a speech recognition system, because obviously if you have a fixed text, you as computer scientists all know that tracking yourself in it becomes a whole heck of a lot easier. So this presents a unique situation. We have basically delivery of a fixed text. Can we deliver subtitles that are sort of coincident with it, and sort of from intelligent subtitles that are truly real time, not like some at homeworker in Ohio, typing several seconds after the fact but actually delivered like really smack on time. So the purpose of this concert is again it's half an hour long. It's to deliver the text of the Ersunada poem precisely in real time in synchrony with him. It's basically got a kind of score follower and a syllable detecter connected together listening to him and attempting to connect to those to propel dynamic typography. Here's the basic scene. He's delivering this. There's a camera looking at him. The software turns his image red so I can superimpose text in a sort of clean way. And I'm here because my speech recognition system and my sort of hacked together score follower and so forth is 96 percent accurate, which is great except in half an hour it gives you four percent inaccurate. And very, very rarely Yop may miss a count. He may do something 13 times instead of 14, but more often than that the system sort of jumps ahead or loses its place. And I'm there to kind of keep the lid on things in case things get out of control. Here's a quick clip of that. [Noise] This is live real time. [Noise] So we can put the lights up a smidge. Thanks. That gives you an idea of sort of what I would say the opportunity -- I'll come back to this later -- to not do a parking lot demo. I'll come back to this idea later. But it's a phrase from one of my mentors Michael Naimark, who I'll talk about. When you're in a research environment, you can just go out to the parking lot and shoot something. Or with maybe 20 percent extra cost, you could actually use really interesting content that kind of produces gripping results. If you fly out a little further than your parking lot and collect material that kind of moves people in a more provocative way. It means the demo can live on in people's minds and maybe reach more people and also in some sense present higher quality challenges. So this is kind of the opportunity presented by this particular wonderful poem. Some work in full body play is an area I've been working in for a long time. Actually recently uncovered a video of myself and Patty Mace's group in 1991, Steve, shows myself interacting with blobs. I'm certain you must have been there at the time. But another time perhaps. Here's just some quick experiments in full body play. This is 2005, me experimenting with a kind of skeletonization. This is not doing a Kinect-based skeleton but rather a video-based skeleton using the media axis transform, to think about a way to anonamyse people and maintain the beauty and articulate quality of their movements in time and space. So that is looking at this kind of motion of the human skeleton as literally computed using a skeletonization algorithm. Here's another piece I'd like to show. It's called the interstitial fragment processor. And it depends sort of remembering that in the arts -- my mother's a painter and she, when I was a small child she said always look at the negative space. Never made any sense to me. Somebody nodding his head. But the negative spaces are these shapes that are formed in and around and in between the body, as shown here. And when I finally -- when I finally understood maybe at the age of eight or nine or ten what she meant by negative space, it was really an interesting awakening for me. And something that stuck with me for a long time. So here in this project called the interstitial fragment processor it processes the interstitial fragments. It's just like that. It finds them and people can begin to discover the spaces between them and I want to say almost savor them. And I think that's what starts to happen here. People even in some sense get to meet each other because they're savoring the spaces between them. So it gets really chaotic after a while. There's kind of an audiovisuality to this, making them physical, making them fall down, these are folks that don't know each other but they suddenly discover the space in between them. And they also realize they can do it without touching each other. They can do it just with the shadow. This kind of shadow play at the same time as making us aware of things in the world that we don't normally see. We forget these shapes are always around us. Okay. Faces and eyes. How many of you have been to the Exploratorium in San Francisco? It's one of those life journeys. If you haven't been you have to go. There's actually a place in Switzerland it turns out it's even better they make sure nothing is ever broken. That's the Swiss. When I was a kid I visited the exploratorium and I saw this project there. It's a simple mix and match game. You guys got it right off the bat. But you understand what it does. This thing kind of lasted with me. Always been interested in faces. I think as artists we've got a lot of, I draw a lot. I draw people and I just have a memory for faces. So I've been interested in faces for a long time. And in this case I wanted to be able to do exactly that. Do it with recordings of people. And particularly with video. So this project creates this kind of mash up of yourself and your peers. It's in some sense creating a generative group portrait of people from the project's locale. Made with Zach Lieberman. This project, each of -- obviously you're entering video into the system. You get kind of mixed around with the people who are previous to you in the space. And it's art made by blinking. This depends on the standard open CV face tracker, good old Vio Jones knock off built into open CV. Once you have the unfettered face input, you don't have to put your head on a special chin rest and so forth, and you can do it unfettered capture. Once you know where the head is, you know where the eyes are, and you can use those to do the edits. So the moment you blink, you create a new set of combinations and you also finish or terminate the cut you were working on. Here it is at the San Jose Tech Museum of Innovation, kind of a big installation there. And here it is in high school in Anchorage. Over in the library they have a capture station. We made a special interface for it so you wouldn't have to be out in the cold where all the faces from the people who were inside the high school are sort of brought out to put a face on the building. Now, it's really diverse high school. People of every single color, shape and size. And this simultaneously sort of brings their identities out to the exterior of the building while simultaneously kind of anonymasing them and not making them feel like maybe their privacy is being violated. It's only been one-third violated. It seems to be the threshold, I guess, because it's people can contribute to the system and then this literally puts a face on the building showing the diversity of the people who are in there. I mean, how am I doing this? The Viola Jones face tracker. Better than some of the other face trackers that have come out. You guys have a good one right now. But this is old technology. None of what I'm doing is really rocket science. I'm one who picks up the pieces that fall out of the research labs and say how can I use these in interesting and provocative new interesting ways to create new kinds of experiences. Robots are something that is one of the main reasons that I was attracted to Carnegie Mellon. How many from CMU? Couple. Cool. CMU was a big attracter to me because of the robotics there. And I hadn't really done robotics before but it's a real ass kicker of a discipline. Very comprehensive as you can imagine with software and hardware. Things fail all the time. And sometimes hard to locate where. But I was attracted to robotics, and I wanted to come to Carnegie Mellon to get into robotics and art. In particular I've been interested in eye contact to try and see if there were systems that can interact with us using gaze, which is to say the way we look at people. So this is a robot I made. My first attempt to break down eye contact into the simplest possible situation. It's a robot with a single eye that engages people in sort of a variety of sort of familiar psychosocial eye contact behaviors. Here's some of the guts. Has just three degrees of freedom, up down, left, right, blink. And a video showing what that looks like is here. And so what does it do? It's got a couple of cameras. This is before the Kinect. It's actually trying to do depth from stereo. And from a distance it tracks your body. You get closer it finds your head and looks at you. You get closer it finds your eyes and looks at them. If you get close enough it's about 18 inches it will attempt to look where you're looking if you look off to the side. One thing it does is to get your motor neurons firing it will blink about half a second after you blink. So this creates kind of a feeling of empathy. And in terms of the language of the arts, you know, inverting condition of spectatorship saying who is looking at who. It's something that really is happening. You do feel like it's watching you, and that's the feeling I wanted to get. That one's maybe a bit creepy, this one is a bit more fun. The little robot. This is the big one. And this is kind of a poetic surveillance system that I developed using what I describe as a quasi discarded robot arm from Detroit. In 2008, the car companies were hurting. They needed cash. And they were getting rid of their robot arms, cheap. And so we managed to get some robot arms over at Carnegie Mellon that were basically like dirt cheap, and began to think about using large robot arms as artistic material. Poetic system big brother half Sesame Street. It's a robot that tries to present the feeling that it's constantly surprised to see you. So it's called double taker. It does double takes. And for me it was the problem, the true research problem for me apart from the engineering problem of how the heck do I control this thing which turns out to be a wicked pain in the butt, apart from that is actually the problem of real time character animation. Right? Where you're not making scripts that sort of say, and you don't have any kind of like foreknowledge of what someone is going to do. It has to be real time to produce allusion of a character, allusion of life. Well, so it's looking at you and it kind of is checking you out, looking you up and down, and you're like what is it, my feet? My hair? And it's kind of looking at you and it will look away and it will kind of look back like maybe it missed something the first time. And this is trying to again produce the experience or the illusion or feeling in us that there's something in all of us that's worth a second look. Here's kind of a quick video of that. It shows a bunch of children on their way to, this is unscripted. They do this. On their way to Sunday painting class at the Center for the Arts. So that's just the character it has. It does a couple of other things. If you bounce up and down, it will sort of mirror that to again get the sort of feeling of empathy. Okay. Moving on real quick. That's a quick survey of some of my work a new media artist. What do I teach? I teach studio art courses in computer science or arts engineering in which the medium is self-written software but the objective is culture work, art design, provocation, social critique or innovation in ways we don't yet have names. My students create things like information visualizations. This was a really nice project by a student showing albums that had been sampled primarily '70s funk and albums that had samples in them, primarily '90s rap and connections between them so you could find out for any given album which ones did it sample and any given album in '70s which one sampled it. You can actually see this kind of, this reference generation. If you were in the early to mid '90s you were probably sampling from the early to mid '70s there. Another visualization made by a student. This is actually a detail from a much larger visualization of a student who used a high speed camera and computer technique system to isolate and record every water drop from a five-minute shower. Like a thousand frames per second camera, and behind a piece of glass. And what is the purpose of this? It's pure poetics. But he counted them all and isolated them all and photographed each one. Another project, my students work with digital fabrication tools. This student has written a program to generate cutout patterns for inflatable structures from any 3-D model such as the Stanford bunny. This is a cake decorating machine. It's a pick and place for M and Ms on icing. This is full body interactive work involving experienced design in generative form. The student project grows trees on your upper edges if you hold still long enough on the shadow. This is a nice project by a student. If we could kill the lights for a second. We'll have a look. Interactive augmented reality using computer vision in which the students made a program that interprets a drawing as a control panel. So you sort of do a -- here, you do a drawing sort of like this. And enter play mode and interprets it as a control panel. It puts out OSC, which is a common protocol for signalling between systems and new media arts, and so you can use this to control synthesizer or anything else that he likes. So he's got knobs and sliders and switches and so forth. Just from drawing it. So this came from him from a childhood experience he had of drawing bogus control panels on cardboard boxes when he was small. for big computers. Sort of pretending that they were control panels Okay. So this is the kind of work my students do. My courses often contain students from nine or ten different departments, including art, music architecture, robotics, HCI, computer mechanical engineering, every student writes code without exception regardless of their background. Although I do encourage students to write code in their environment of choice. So it's a real zoo of languages. I have everything from like flash to Matlab, kind of in the same environment. The arts oriented students learn the skills, the arts engineering skills that you might call computational thinking in order to implement their ideas in real working forms. The students from the engineering and sciences encounter something very different altogether. They're invited to apply their skills to questions that are driven by their curiosity, their passion, their senses, even their politics, to problems which may be more personal, more speculative or more whimsical, whose solutions are less provable. They're invited to have a point of view. Some of my students from the science and engineering have never been asked what would you like to make. And having this liberty is the rare privilege of being in school. For some of them it may be the last time they're asked that question for a while. So I try and get them in contact with this kind of part of themselves where they have a dialogue with what they're interested in, because some of them won't be asked that question again. I wanted to give one actually sort of interesting concrete example of this. So the Kinect came out in approximately early November 2010. In February of 2011, my classroom became the first class to my knowledge in the world that had Kinect hacking assignment. We had 24 students hacking on the Kinect and making interactive projects with it. The SDK wasn't out for another six months. So we had to use the open source arts engineering toolkits which were the only way that my students could engage with this medium that they were so interested to play with. So these are some of the projects, and I am actually going to call one out in particular, though, because something interesting that happened. This is a pair of sophomores from the school of art. They're programming a two-person puppet, kind of behemoth character, where one person like a Snuffleupagus a horse where one person is the front end and one person is the back. And so her hands are sort of opening the jaws of this monster and the other guy is controlling the tail and the legs back legs and so forth. It was the first two-person puppet with the Kinect. Look, that's not earth shattering, you know, solve world peace kind of solution to anything. But for a pair of sophomores from the school of art who had only been programming and processing which is a flavor of Java for about a semester, this was huge. They got 50,000 views online. For an art student that's a profound thing because prior to this all they had done was they exhibited their paintings and drawings at the local school gallery, maybe seen by 50 people, not 50,000. So it was a totally transformative thing for them to be able to use skills they barely had in Java programming to be able to engage in culture make something that 50,000 people would think was interesting. That was made possible by your wonderful invention, the Kinect, and also the new media arts community that came together to make it possible for people to interface that with these open source programming languages and environments that would allow these kids to make this kind of thing. Okay. Some work from my lab. As I mentioned, we support atypical and anti-disciplinary work at the intersection of arts science technology and culture. We've run a few conferences among other things concerned with providing basically bolstering computational thinking. Microsoft Research. You may know, has invested, thank you, in the Carnegie Mellon Microsoft Research center for computational thinking which supported several events I've run that are basically attempting to examine how people can use passion-driven creativity-driven visually-driven objectives as a way of bolstering computational thinking. One of these was the first art and code event programming events for artists and the rest of us. This is a crowd -- this is not staged. These are people ranging in ages seven to 70 learning to program in Java in this case, because they suddenly find there's a reason for them to do so and they can make these kinds of interactive visual experiences. The third art and code conference was held in October of 2011. Stewart was very kind to come out. This was art and code 3-D. DIY 3-D visualization sensation, this is the event that was actually going to lead to the work that James is going to show tomorrow. We wanted to have a Kinect hacking event and bring together all these people you surely saw the Kinect hacking phenomenon happen online. We wanted to bring these people together working on different environments different objectives get them to talk together. Now, I said DIY 3-D sensing and visualization. We did talk about other things like light R scanners and structured light and other kinds of approaches to doing 3-D scanning. Look, in late 2011, DIY 3-D sensing and visualization pretty much means Kinect. And we had a lot of different approaches to how to use the Kinect in kind of interesting artistic ways. You have a question? Sorry. I thought you raised your hand. Okay. So this was the world's first Kinect hacking conference thanks to you guys for support. And it actually had a lot of interesting outcomes one of which probably the major research outcome for us was the project that James is going to show you tomorrow. And I'll show you a fragment of right now. So some research from the studio. Could I ask you to hit the lights for a second. This is sort of a forward pointer to what James is going to be showing. James has been working with Jonathan Menard a trained documentary in film making on Kinect-based cinema. What it really allows is for camera work to be something that's done in post production. It allows you to tell a new story or emphasize aspects of the story later after the shooting is done by changing camera positions around a virtual object. Another way of thinking about it is that it makes video captured people as plastic and as malleable as computer generated characters from CGI. So suddenly you can have virtual cameras that can move around these subjects and also serve play with the data that constitutes the subjects in interesting ways. Here's a quick clip. I'm not going to ->>: Basically explaining how the database works and say as one point here and then you go around and there's another, another one there, another one there, because this is close, this is the one which is matched. Explains everything like that. >>: Is there an experience a digital experience you desire to have? >>: Yes, transportation. >> Golan Levin: So what you're seeing here is a documentary that James will talk more about tomorrow. It's called Clouds. In case you don't have a chance to see it or in case I want to give you a forward pointer to what it is it's Kinect-based cinema involving interviews with globally renowned creative coders, globally renowned media artists who work with code as their material, also curators and critics as well thinking about new media arts and it's a series of interviews. It's going to be an interactive three-dimensional generative documentary that James will be talking about. James will also talk about the techniques of RGBD cinema which he's developed in order to make this kind of thing possible. I think you'll agree it doesn't quite look like anything else that's out there. If you've seen anything else like it that's out there it's probably because they're using James' open source toolkit which now has been picked up by a lot of other projects. So that's one example of work which I'm very proud to have come out of our lab and proud to have supported in its earliest stages. James and Jonathan met at the art and code event, Microsoft Research supported. Another project to kind of give a picture of the diversity of work that we do at the lab, this is a project called the new artist. It's a set of robots. And it's concerned with the question of can robots kind of observe each other and respond to each other. One robot, this one here, is the performer. It's attempting to ingratiate itself with the observer. The observer judges the performer and decides whether it thinks the performer is boring or interesting. There's your word interesting. In this case, the performer is dancing around, as you'll see, hopping around, and if it's too repetitive, then the observer, the audience robot will get bored. If it's too chaotic, the audience robot will get bored. The audience indicates pleasure or displeasure by its posture. Sort of about they're sort of in feedback with one another. Here's a quick video of that. So there's the performer. There's the observer. And they're each looking at each other to kind of gauge how they should respond. This is kind of sweet spot between complete repetition and complete chaos that the performer's trying to hit. So it's a completely cybernetic system made entirely with robots. There's no person in the loop there. And a little bit of progress towards Lego locomotion as well. He's trying. Okay. See. Look at me. All right. Lastly, the last project I'll talk about from my lab, we're working with the robotics institute which has invited us to produce a set of artworks that are accompanying their mission to the moon. They're trying to win the Google X prize, $30 million attempt to get to the moon. And so they invited our lab to produce a set of artworks, one of which, for example, will be using the rover to draw tracks on the moon. Some of which will maybe line up anamorphically like so. We hope that we can produce kind of a drawing program like an applet on the web where kids can go and have drawings and have the drawings rendered on the moon. We'll have to innovate some crowdsourcing way of collecting a million drawings and have people vote on a million drawings and not only have penises. [laughter] You can imagine. All right. Part two. Now this part's a mess. But I just want to say that as you well know, computers in, I'm going to do this quick really just a few things I want to say about artists and research environments. Computers in the '50s through '70s were really expensive. Dedicated to doing important things. So if an artist wanted to come along and use a machine like that, it was pretty difficult. Manford here, still alive in his 70s, he still programs in Fortran of all things what is renown for doing early sorts of generative artwork, quasi system you would make the plotter work, to be able to do this he had to beg and plead his way into using the mainframe on Friday nights at three in the morning. And especially because it was the late '60s, and they're like long hair, what the hell are you doing here, it was a big significant cultural divide. You want to use my computer for what? Typical thing. Same with Lillian Schwartz who in the early '70s fought her way into Bell Labs. She kept showing up, innovating heavily in the area of what became computer film, computer graphics. To make computer graphics at the time, it wasn't really a priority for Bell Labs. They were concerned with more important things. Nevertheless, a real pioneer in the area of computer graphics and computer cinema. John Whitney by contrast in the mid '60s was actually invited to be an artist in residence at IBM. 1966 through 1970, he created IBM a wide variety of early computer art, sort of generative and sort of geometric computer art there. He did not do his own programming. They assigned an engineer to work with him. And as I'll talk about, one of the -- two of the big sort of models you can have for artists in a research lab context are the sort of exotic zoo animal versus sort of free range chicken. In this case he was sort of the exotic zoo animal, expected to produce these things. This is an exotic zoo animal amiga commissioned Warhol to be an artist in residence. The idea here objective here was to bring some cultural cache to this hardware they were trying to push on the world. And you can find this video on YouTube. It's pretty funny. Now, one of my mentors is Michael Naimark, and in collaboration with the architecture machine group and a variety of others at MIT in the late '70s and early '80s, he created something called the Aspen movie map. Now, Michael has been an artist and researcher invented, interested in representing places for a long time. In addition to his influential work exploring virtual reality he's known for his advocacy of media art as a stimulus for technological innovation, having directly helped establish a number of prominent research labs, including MIT media lab, Atari Cambridge media lab and Apple multimedia lab, interval research. In the late '70s he was a fellow at the center for advanced visual studies at MIT and there he helped create this landmark hypermedia installation allowed visitors to navigate the roads in Aspen, Colorado. The movie map was made possible through an artistic abuse of the world's first laser disk player, namely by taking a device intended for the storage and playback of movies and instead using it for random access under interactive control. That was an artistic idea that he brought to the table and the team helped create. If you had said to people in 1982 that some day you'd send cars to every city in the world and record video of every single street in the world, that would seem ludicrous. But you can see here I by no means saying they ripped him off I'm actually rather saying this was an influential project that was shown at a lot of different places including early SIGGRAPHs, and it initiated a conversation. I would say it had a cultural influence that many people understand what it would be like from an interface perspective to be able to walk around and have a user-driven tour of virtual places. It may not have seemed possible or even something like you'd want to have. And in fact there's a fairly interesting story. The Aspen movie map was funded in part by a federal research grant. In 1980 this project was given the dubious golden fleece award by U.S. Senator William Proxmire, a sarcastic recognition he bestowed on projects he felt were egregious waste of taxpayer money. So I'm just trying to show an example of how artists in research environments can help transform the language, the dialogue, the ideas that are discussed. How many of you guys know this one on the left? This is Myron Krueger's video place. Myron Krueger was one of the first artists to work not only with computer vision but with interactive media, interactive computing at all. From 1969 to 1974 he developed a project called video place, basically silhouette-based interactivity. He's a bitter man. When Sony contacted him, he wanted numerous arts awards for this, and actually was shown, for example, won the Golden Ike, the world's largest most significant prize for electronic art in 1990. When Sony contacted him in early 2000s saying look we're planning on having a camera based gaming system become the eye toy, we've heard you thought a lot about body-based camera based interactivity, would you like to consult with us, Myron made a mistake in my opinion. He said you know consult for you? I'm going to sue you. And it was one of those things that -- it really presents a difficult challenge, I think, for new media arts, especially when one can I think not -- without unjust cause actually say that one's kind of invented a kind of scope, scope of ideas. It's a problem of the medium. Okay. Briefly. Here's some reasons why you might want to have arts in your research lab. From Stewart Brand in his article Creating Creating, appeared in issue one of Wired 1992. He provided the following reasons to have artists discussing his time at the MIT media lab. He said the deal was very clear. The lab was not there for artists. The artists were there for the lab. Their job was to supplement the scientists and engineers in three important ways. One, they were to be cognitive pioneers, exploring the horizon of the newly possible. Two, and this is maybe probably the worst reason to have artists in your research lab, but not a bad one, just the worst one, we're to ensure that demos were done artfully. That is to say with presentational craft. It's the least interesting reason let's say to have artists there. Three, they were there to keep things culturally relevant, having real artists around was supposed to infect the place with quality, which it did. Michael Naimark who made the Aspen movie map I mentioned provided six reasons in 1999 article which was a speech actually introducing exhibition of interval research artworks at the San Jose Tech Museum of Innovation. He described the following reasons why interval integrated new media arts into their everyday research activities. He said: New media arts projects, one, provide stimulation and provocation to our research community adding meaning, entertainment and emotional resonance to our work. Also they often act as magnets to bring together unconventional combinations of skills and talents. And three they can also provide content to test tools and even tools to test content. Another set of reasons. New media arts projects are sometimes means for collecting valuable data. Both through explicit query as well as observation. May leave researchers down unforeseen paths and result in discovery and intellectual property. And external deadlines and public scrutiny which can force for decision-making rigor and completion. They keep us street smart. Putting on a show as a testbed for new ideas. A simulation of the real world. So some reasons from, Leah Buickly, professor at MIT media lab, she points out that research requires diverse modes of thought. Artists often think in ways which are concrete rather than abstract. They work in ways that are improvisational rather than planned. Create things that are expressive rather than functional. I come from Carnegie Mellon. I work with engineering computer scientists all the time. As much as I love my school, the computer science department can sometimes enforce a monoculture where if you're making things which are expressive, let's say, rather than functional people are saying why are you doing that, what's the use? So that already begins to cut off certain modes of inquiry. And it may be premature to ask that question. Right? So that's an example of what I mean by sort of having a diverse set of modes of thought. John Mida, my thesis advisor, now the president of RISBE, talks about STEM to STEAM. Trying to integrate the arts into the Stem fields. He says artists and designers are accustomed to innovating with heavily constrained resources. That's kind of a good thing. They're used to dealing with ambiguity and multi-valent perspectives and they're able to deal with adversity and volatility, noting that the future world will be warmer, more resource depleted, more volatile. Mida notes: With greater despair will come greater hope and greater art. With fewer resources will come greater ingenuity and greater design. We know the world will be volatile and uncertain. We know there are no better people to handle that kind of world than artists and designers. Lastly, artists can bring critical perspective. And this can be the hardest to explain and the hardest to see the value in. But artists can bring criticality to an environment otherwise vulnerable to the Achilles' heel, the cultural Achilles' heel of techno enthusiasm. They can help predict shots from the outside. Chat I had with Michael, he observed this example. He was sort of saying to himself, if you guys know about the Google Street View Wi-Fi scandal caused them to have lawsuits in every single country around the world. Huge PR disaster for them, where they were collecting information they shouldn't have been from the Wi-Fi. He said, look, if they were someone culturally savvy on the inside who also had the trust and the technical respect of the engineers there, this possibly could have been avoided. He's thinking of himself. He's someone who really understands about representing places like street view like situations. And what he could have done had he been on the inside there if there were that kind of thinking in place. Well, look, there's two basic models you have exotic zoo animals like Xerox pair. This was the Xerox park artist and research, artist in residence program. Artists would keep the work they would do. Then there was the free range chickens model I mentioned where the farm owns everything. And here in this case artists were there to participate in the cultural life. They weren't artists in residence. They were just researchers just as some of you happen to be anthropologists or scientists or engineers. I'm sorry. I've gone long. I'm going to wrap it up there and just say artists outside the lab are starting to meet their own needs and they're routing around limitations by hacking your protocols. And making their own arts engineering environments, processing Arduino open framework Cinder. These are open frame arts toolkits they developed for each other to easily sketch and deploy quality interactive systems these are not commercial products. They're generally fewer than a dozen people developing each, often even fewer, but there are hundreds of thousands of other artists and programmers who use these. The people who work on these environments do so out of love not money. And I actually wanted to say I think probably one of the biggest opportunities you guys are missing, if I could give you guys one piece of like: My God this is such an opportunity you guys are missing here at Microsoft, like oh my God, it would take approximately one person halftime to be responsible let's say for making sure that open frameworks worked well on the Windows platform. There are 100,000 people using this to innovate in high tech ways. If there was somebody within Microsoft who was lurking on the open frameworks list whose job it was let's say halftime to do so and to make sure that it was a pleasure to develop open frameworks applications or processing applications in the Windows operating system environment, in the Windows ecosystem, you would be investing in the interest of 100,000 incredibly culturally innovative, culturally sensitive developers, different than your C# .net developers these are different group of people. These are people. Yes, question? >>: The challenge with that -- I agree 100 percent with what you're saying, is that direct X versus open GL right now. And with Windows 8 it's optimized for direct X which provides features like stereoscopic out of the box HMI which isn't ->> Golan Levin: But it's an offline discussion we can talk about. But two quick responses to that. The first is a give and take. I think that open frameworks environment has no prima facie objection to direct X. I think to have -- to say oh you're working on Windows instead of Mac or Linux you have the additional option of having it work well with direct X. In fact that would be a gift to the open frameworks community because now we have direct X as an option as well. Right now there's nobody who is stewarding that. It's not that they reject direct X it's no one's stepping up to make sure this 100,000 strong ->>: Talk to you off line about this. The open framework forum is a little more embraceable, DL Cinder came on their forum, saying we would love direct X port but we haven't had the context. >> Golan Levin: James has a lot to say about this too. He's deep in the community as well. There's an opportunity for you guys to make sure that the Windows ecosystem is like loved by these people. And currently it's not yet happening. I think it's also a give and take, too. You guys still make Office for the Mac, for example. It's also to say, okay, look even if we're, even if we don't have a whole lot of love for Java we could still help processing out. Even if we don't have a love for open GL and it wasn't made here, we could still support that so that people who are working in those environments can have a good experience in Windows. Here's an example what creative practice looks like, I don't know how many of you saw this project, but Jason Serage, this isn't Jason Serage, he was a computer vision researcher who released a very good face tracker, kind of a appearance model face tracker out into the wild. He said basically if you want to use it, here's the code. McDonald and Castro picked it up and started to make this interesting application where they said you know since we know where the face is, I could -- since I know where his face is I could do face substitution. And I could very easily then substitute Arturo's face for mine. Or I could substitute, let's see who's next, he's wearing Arturo's face he's saying. I could be Marilyn. Now, Kyle -- okay. Busts this out in a day. He busts this out in one day based, of course, built upon Jason Serage's face tracker released out in the world using open frameworks, open frameworks powerful gives artists designers leg up busting out interesting kinds of executables. He puts this out there. Okay, is this research? The question here is this is what I mean by cultural innovation. This is the concept that he puts out there in the wild. You're all smiling because it's good work, and he gets a million viewers in about, in less than a week. He gets like 200,000 viewers in a day and a half. From this kind of work. This is the kind of innovation that represents an approach to hacking outside of the lab now. This is what I mean by people who are hacking outside the lab. Something else happens as well. Not only does Kyle make this fairly provocative demo of what can be done with the Serage face tracker, he does something else as well. He wraps it up in another project he calls face OSC. Remember I mentioned OSC the student who made the control panel who speaks OSC. The OSC is this open sound control protocol for sending data around. He made it so that the face data could be sent over OSC. And suddenly a new ecosystem arose where hundreds of other people were using face OSC to make their own face applications with processing and open frameworks and maximus P and super collider and other kinds of ableton and other kinds of artistic environments. This is now what creative practice looks like outside the lab. He makes interesting work and he also makes it possible for other people to make interesting work. This is what our spaces look like. This is what our laboratory looks like. This is the open frameworks, ours electronic 2008. Lastly one funny story. How many of you know this patent? I'm on it. It was from interval research. Have you guys been following Paul Allen versus the world? You have been. This was an idea that myself and some other folks had at interval in 1994 and 1995. It was just the idea that you could have a kind of agent, software agent that would do Internet searches in the background while you weren't paying attention. This seemed kind of revolutionary at the time because usually you'd have to make an explicit query. I'm searching for such, you would get a response. This agent making requests in the background, wrote it up. It's attention manager for occupying the peripheral tension of the person in the vicinity of a display device. Whatever. It turns out that as some of you, half of you who have heard of interval know in some sense nothing came of it. Well, this did. And a couple of years ago Paul Allen realized it came out of interval. He realized everyone had been infringing this, and he proceeded to sue AOL, Apple, eBay, Google, Netflix, Office Max, Office Depot and YouTube, a large company he's not suing which is pretty awesome, I suppose, although he conceivably could have what's interesting is that actually there might be a very, very large set of [inaudible] and a large amount of revenue might come from that patent. Here's the interesting thing, everyone on that patent they were not the engineers and scientists. It was all -- it was the folks from the humanities and the arts, all the folks who were on that patent were basically -- it was the journalists, the artists, and a few others who were basically like culture workers. So I guess the point of my story is that these can come back to you in unexpected ways, in the $500 million patent lawsuit that you win several years down the line, maybe. Anyway, it's just to say that sometimes unexpected things can come from having these folks around. It's late and I'm so sorry this has gone so long. Like I said, we had that risk. I'm really grateful to be here. Thank you so much. [applause] I promised you really should see James' talk tomorrow. an hour and a half. >>: Same time, same place. He won't go for Are you open for a couple of questions? >> Golan Levin: If anyone has to go you should. I realize it's late. If anyone doesn't have to go, I'm happy to take questions. Sir? >>: There's a lot of people [inaudible] [inaudible] so thinking about why research institutes like [inaudible] artists because of all the risk you're talking about. But the artist, why not should go to and come to Microsoft or to whatever. If you have access through open framework processing, some tools state of the art, why is it in or not? >> Golan Levin: Kind of a candy store, really. I know artists who wouldn't. And I know them who wouldn't because they're really committed to maintaining sort of cultural and financial independence. They insist on working in open source ways. And they say if I can't share what I'm doing, if I can't share the code I'm doing with the world I don't want any part of your system. There are other artists who recognize that by coming here, they get a chance to have early access to some incredibly exotic technologies, and explore and be the cultural pioneers, not the science pioneers necessarily but to be the cultural pioneers of exploring the cultural possibilities and artistic possibilities of the high end stuff that you guys are developing. So that's what appeals to them. It's certainly what appeals to me. Now, they want to be able to have a chance to explore and play. And also a chance to participate, because it's not just like, oh, can I use that thing you made, that was really cool. But also to actually participate in teams with you folks on determining a technological agenda because they have something to contribute to that. I hope that's a good answer. James, do you? >>: I'll contribute my own thoughts tomorrow, too. in. >> Golan Levin: It will be wrapped Yeah. >>: I'm chewing on the notion of artistic and novel. A number of the things you showed were really cool because you were the first one of anyone who had done it. And conversely there was a thousand years of people using oil paints who were not necessarily doing the first cool thing with oil paint but doing something that touched us or brought new vision to us or opened our eyes. >> Golan Levin: That's a great question. For the last 100 years, there's been a certain string of the arts which is absolutely preoccupied with continual surprise and continual novelty, has an Achilles' heel, of course. Nothing gets old faster than novelty itself. things. At the same time it allows you to leapfrog a few different It means you can quickly become the master of something, because there's no one else to compete with. It means that you have novelty to be able to show something in the world and people say wow I've never seen anything like that, and you immediately get their attention. And so those are some things which absolutely have appeal. At the same time, actually Michael Naimark who I mentioned a few times in this talk, has a great article, First Word Art Last Word Art. Artists who want to get in the first word. They want to piss on the territory. They want to basically say aha, I've claimed this. There's also artists who want to get in the last word, where you produce such a definitive contribution to an established forum that people say wow that was an amazing symphony, in a mature sense. It's the difference between like Brahms. And he uses like Brahms and Beethoven as examples. Like you get in the first word, you can sort of define the form. You get in the last word, you can really make a profound statement and take advantage of all the stuff that's been figured out. There's different ways of working. And they have risks. >>: I guess what I'm thinking the flip side of the Aspen project it was odd. Today it's Google Street View. >> Golan Levin: Yeah, in the same way that we forget the identities of the people who, the artists who made gothic cathedrals. It's like we forget. Because now we have these cathedrals that's a point made by Stewart Brand there's a shifting territory. If you're not concerned with calling yourself an artist but call yourself a cultural explorer it's a lot easier to handle that. >>: It felt like things that technology. So many of them shadows. It was just one of of it. But the rest of them like a big clunky table that thoughts on what can happen? were constrained by the display I was looking at projectors which cast them. The shadow one which took advantage they were a pain in the surface. Looked you couldn't sit under. Do you have any >> Golan Levin: If you have something cool I can take it. As an artist you have to deal with the inherent properties of any medium you're working with. And those properties can be problems. But hopefully if you're a good artist, good designer, you can make those not foregrounded, you know. But your sensitivity to those issues is a testament to your own understanding of the problem of being an interface designer and making lovely craft. It's the same way that like for years we had these huge monitors on our desks. It wasn't that long ago, right? Sumit? >>: The question similar to Danielle's, but I think it's we encounter as researchers a lot, too. But when you're out there doing new and interesting things, often you realize years later that you've made the influence some technology but it's through some unusual path, and the law back to you might not be clear, and that could have spiritual ramifications. >> Golan Levin: Yeah, some people are real bitter. >>: Some are bitter. I'm wondering like when you talk to your students about -- like this is going to happen to everyone, right, if you're going to be out there in the world creating new things. And what kind of advice would you give to people to help them have the positive path. >> Golan Levin: This is a huge conversation. This is like we're on a mailing list together where this is like talked about usually like once a month. It's a huge conversation. The real winners are people who cannot only get the first word in but also get the last word at the same time. Let's just take one example. It might be Sergeant Peppers. There's an album where there was nothing like it before and nothing like it after. And to have a level of craft and commitment and purpose and focus and passion and statement where you create something that people go wow, that was an album that we won't forget, you know, it sort of defines a new medium and sort of closes the book on it. And that's really hard to achieve. That's a real consummate kind of expression you don't see very often. Otherwise there's plenty of people who get forgotten. >>: But seems like there are people, when you were describing stories, there's some people who can kind of, who are able to look 20 years down the road and I was part of making that happen even if they're not named per se which seems like a much healthier ->> Golan Levin: May be best to have a Buddhist attitude to everything. That said, no one wants to get ripped off either. >>: Exactly. >> Golan Levin: I feel so bad it's late. I'm happy -- please. >>: You seem a little bit isolated from usefulness to ->> Golan Levin: That's other people's problem. >>: I just feel maybe there's a way, actually could glom these three elements together. >> Golan Levin: I think that life is short and it's hard to hit all three at the same time. So I mean, for example, a lot of my work has been people have said I want to use it with children who have disabilities. Like this could help people learn how to speak or could help people who are disabled in terms of giving them -- that's a real -- actually it's just quality of life again but quality of life that makes people feel valuable and feel like they can have an effect in the world. That's useful again. I try and do that. But it's hard to hit all three things. So James, you ->>: I would add that there's been lightened self interest in making things interesting beautiful. And is it useful. That's beautiful, there's the trichotomy you put forth. It doesn't have the dollar sign attached necessarily. And I think even touches on what we were talking about before, which is you get value in knowing that you push things in a direction that you think the world should be and that's your reward, is seeing, if 20 years down the line you see it productized or turn into something that doesn't acknowledge your influence on it. It's like I helped shape this. So it's the Buddhist attitude. But that is useful. >> Golan Levin: much. [applause] I have to let you guys get to lunch. Thank you so