Reality under Construction
The Norwegian dance/media performance company Kreutzerkompani is operating with
two levels of reality that is being augmented into each other. In the beginning of the new
millennium they created a number of media performances that all had one main interest:
To make use of live performance and real-time recordings as creative material on the
same level – so that the performing and creative processes are melting together into one
practice. One example is Sync #2, where a dancer is performing live in front of a public
and being recorded real-time on video at the same time. The recordings are sometimes
synchronous, sometimes stopped or repeated in patterns on a big screen behind the
dancer. The camera is not only recording the live-performance but is recording the
recording as well the earlier recordings being played back. The result is a mirror effect,
where the image is repeated indefinitely into seemingly parallel worlds – but with the
added element of recorded material being projected onto the live performance.
The sounds are prerecorded material combined with real-time recordings from the
dancing, the audience and the “ambient” sounds of the entire space played back onto
them.
In Sync #2, reality is under construction. Furthermore, reality is not being constructed from
the “outside”, or through physical phenomena only; it is as much constructed by
conceptual integration of ideas, actions and technology.
Working from the notion that augmented reality in contemporary art is a reality und
construction, I will attempt to investigate augmented reality in contemporary art with
examples from the practices from the Scandinavian art scene. Moreover, I will investigate
augmented reality as part of a transdisciplinary field of art production that is taking place in
the expanding field of new media.
However, my focus on augmented reality will not only be as a new media issue, but as part
of a new paradigm of artistic production in public and social spaces, combining physical
experience with technological interface. Hence, the augmentation of reality in
contemporary art is not about aesthetic transformations alone – it is not a matter of form or
style; instead, it is part of a cultural change involving the relations of art practice, art
institutions, humanistic research and the public space which is brought on by the refining
and expansion of technology into every aspect of our lives.
Above all, augmented reality is about how experience interacts with knowledge; and how
that interaction, in turn, creates interfaces between public spaces, social spheres, and art.
In computer science the term Human-Computer Interaction, HCI, is used when studying
this from the practical side. In the 1980s and 1990s, the notion of the embodied mind and
conceptual integration as a structuring principle of knowledge was formulated by Mark
Johnson (The Body in the Mind, Chicago 1986) and Mark Turner (The Literary Mind,
1999). In the last decade, the idea of the embodied mind, bodily based rationality,
cognition based upon physical and bodily active relations with the surrounding world, has
entered the stage of artistic as well as aesthetic research. Paul Dourish, in Where the
Action is – the Foundation of Embodied Interaction wants to nourish the perspective that
sees “the embodied practical action in the world as the foundation of our conscious
experience” in this kind of research. Augmented reality in contemporary art is a reality as
well as a social aesthetic under construction; it is an artistic research into the question of
Human-Computer Interaction, the function of conceptual integration, and the embodied
practical action of art and art experience. It is a structural combination of circuits that
traditionally has been divided: The fusion of the performing mind and a technological body,
of conceptual sensing and media consciousness, and last but not least a public is ever
more active in the creation of (that which we understand as) art.
The Scandinavian scene is very innovative and active, experimenting extensively across
old genres, art forms and disciplines. This experimental focus on production that artists are
conducting qualifies, I would claim, as scientific and research and philosophical inquiries
into a field of production that this article wishes to investigate as an intensely rich and
complex – and extremely important – source of knowledge about the culture we are living
in. In fact, I will attempt to examine this field of production, in a sense, anthropologically –
which means that I will attempt to look at it as a field of cultural production which is
semiotic coherent and structured around the artist as a functionary or indeed mediator
between art and technology.
Instead of discussing art and media as something separate that collide suddenly in new
media, I would like approach an analysis of the mediator from a different angle – focusing
on the research done by the Danish media performance group, Boxiganga. I will attempt to
analyze their “Augmented Reality Project, part 1-3” (1998-2008) and their research into a
relations technology –art as a network of open systems:
We intend to develop relations-orientated multimedia works, which function in
the social realm, in which we as people continually recreate and reinvent our
existence — in relations between people. This is “relations technology” as
opposed to functional or manifested technology; Open systems in which
content and relevance are directly dependant on involvement.
I will then move on to discuss augmented reality within the framework of that which Boris
Groys call “die zeitalter des medien”. Groys claims that the situation of art in a media
culture is indeed not only that of emergence of new aesthetic paradigms, but the infusion
of a new "logic" and "epistemology" into our concept of reality and cultural patterns. Thus,
there is an "amplification" of reality from media taking place in aesthetics as well as in the
broader cultural context.
The challenges of a new media culture and the artist as the
functionary/mediator of this culture, also brings up the question of how media
consciousness is augmenting the modern public/political space. Jürgen Habermas, writing
in 1962, believed that the modern public space was founded in a literary consciousness as
a kind of a common/ shared ground – in the general public as well as in politics and art.
How, then, does the notion of public space, I am very tempted to ask, change if this
common ground is founded in a media consciousness (which it already is)?
The competences of traditional institutions and genres, indeed the epistemology of those
competences, are changing. Instead, we get new domains and new competences – and
an inversion of institutions and the public space. Reality, art and art institutions are under
construction.
Boxiganga: Augmented Reality Project
The fusion of performing mind and technological body is clearly visible in the practice and
artistic strategy of the Danish media Performance group, Boxiganga - Karin Søndergaard
and Kjeld Pedersen. In 1998, they formulated the principles of an Augmented Reality
Project which should create an environment for artistic research into the use of humancomputer interaction in artistic/ performative installations. Building from a tradition of Nô
Theatre and “classic” performance art practice in the 1980s, the Augmented Reality
Project was to be realized in three parts: “Relational Mechanisms” (1998-2000),
“Constructed Interactive Spatiality” (2000-2005), and “A Sensing Sculpture in Public
Space” (2005-2008). The result of the first part of the project, relational
mechanisms, was shown at The Museum of Contemporary Art in Roskilde in
january-march 2000. Working with a network of Apple G3-computers, the basic
principle was to place the computer and data-processing in the background; this is
preconditional for achieving the illusion of reality in the human-computer interaction.
Augmented reality is a construction of physical conditions that should be present in
order to be able to experience a realistic physical relation in space:
In our multimedia set-ups, the computer is relegated to a place where data is
recorded, processed and transmitted. We can then be concerned with
multimedia in a context of Augmented Reality, with creating spatio - sensory,
perceptive and reactive constructs.
An interactive multimedia set-up is a world construct, in which everything
consists of second hand impressions of different forms of processed
transmissions — projections, sounds and reactions as interpretations and
translations in a constructed reality. One is never in direct realization, but in
search of meanings with one´s own physical presence, in which one´s own
senses and actions can conquer, interpret and recognize.
The Augmented Reality Project part 1 is organized in four complex, spatial constructions:
“Smiles in Motion”, “Mirrechophone” (Mirror+Echo+Phone), “I think You — You think Me”,
and “The Different Stories of a Bride and Groom”. Each spatial construction (or perhaps:
augmented installation) play with the notion of constructing the preconditions for how we
are experiencing actual phenomena and relations in physical space, through hidden data
processing. Boxiganga works with specific strategies which uses the audience’s actions
and reaction as a framework for the creation of an augmentation of reality. It is a pretext
for making it possible to experience the construction of reality and by the same token
enables the audience to reflect upon their interpretation of this experience:
In this way, the visitor also becomes involved in an augmenting of what is able
to be sensed and is likewise brought to an augmented state of interpreting that
experience.
In fact, the basic function of the installations often requires that two visitors enter into a
relationship and investigate an interpretation of the contents of that relationship. These
installations then are situations for augmented relationships.
It is through the body´s organs that we sense and act. In this way our being interprets the presence
of people and things and brings a certain reality to that presence. Augmented Reality involves the
body through the installations presented here, and in doing so, proposes “conversations” at the edge
of our normal means of sensing and communicating.
In this project, visitors will come into contact with a series of staged and coreographed, high
technology installations that can sense their presence. These “sensitive” sculptures are likened to
pieces of furniture in a room.
But the installations, each in its own way, don´t only sense, they also react. Thus they promote
relationships through experiences that may take place at the edge of the senses.
The first, Smilets bevægelse / Smiles in Motion, was a
“Spejlekkofon” — Mirrechophone
Institutional redistribution of competences
Historical developments in the 60s and 70s have reshaped art. Some artists have stressed
mass consumption. Marcel Duchamp and Andy Warhol are among the most important
artists. In their art the single object is lost and their works becomes multiple objects
themselves. Other artists have chosen to create art without creating objects and new
forms of art have showed up: performance, conceptual art, processual art, environmental
art, digital art, mail art and more. Interesting is that many of those art forms have the realtime as an important ingredient.
The museum as an idea is old and the origin can be found in the cabinet lockers of the
17th century. The cabinets were filled with different kinds of artefacts that could represent
the known world. The owner of the cabinet could then literary hold the world in his hands.
The object was not just an object, it represented something more, a part of the world.
Many of the central museums were founded on the 19th century and has preserved the
idea of the object as a key to the history. The museums use the artefacts in the same way
as they did in the 17th century. By collecting artefacts we could get an image of the past
and by collecting and preserving them it is possible to preserve our history to future
generations.
Today when objects in general lose the function as holders of status and history it is
natural that the antiquarians at the museums are confused. How to preserve our
contemporary time to coming generations if there are no relevant artefacts? Museums are
going through a gigantic identity crises at the moment. The distance between the past and
the contemporary time is growing all the time. What are real i.e. old artefacts made unique
and by hand craft becomes rarer. They represent something not artificial and will gain
more and more interest. How can we look at these objects? In our post modern time, or
perhaps post-post modern, we know that the object is no key to the past, objects are not
time machines and they cannot give us an image of the past. At their best they can give us
a number of possible explanations filtered by the present time.
At the art museums the identity crises is very clear. They have a mission to collect,
preserve and communicate art works that represent our current time. But, how is it
possible to preserve or collect art works that are built on real-time strategies and are made
for the moment, art where no object can be found and where the concept is what matters?
According to the French Philosopher Jacques Derrida, the Public Museum is founded by
two ideas: First of all the revolutionary and institutionalizing, since the museum creates the
framework and connecting cultural phenomena into (new) identities; Secondly, they are
traditional and conservative since the museum at the same time are preserving, not only
the institutions and (new) identities, but also the cultural patterns inherent in the society
into which they are institutionalizing themselves:
Museums are … Revolutionary and traditional. An eco-nomic archive in this
double sense: it keeps, it puts in reserve, it saves, but in an unnatural fashion,
that is to say in making the law (nomos) or in making people respect the law.1
Using the notion from Peter Weibel (above), I would like to move Derrida’s ideas into the
field of ’competences’ that are in play in the game of the cultural epistemology. The
competences of a museum are encompassing both the revolutionary and the conservative,
and this, in turn, becomes very much the case when institutions are being formed around
the possibilities and new public spaces that the technology behind the term New Media –
or, as I prefer to call it: Digitally distributed environments - create.
However, it is very common to see only the conservative and un-connected
side of museums – which in large part is the fault of the museum’s themselves. The last 50
years or so, after the Second World War, focus has been on conserving culture and art, to
1
Derrida, Archive Fever – A Freudian Impression (1995).
a point where it has almost hidden the other competence of the museum. Perhaps it is
even political incorrect to speak it… but, nonetheless, the distributed environments have
created an unprecedented and (for some) unexpected break with the ‘traditional’ museum.
Museums are an augmentation of a reality, and ways of organize knowledge, that has
been constructed by a democratic society. In that sense, we primarily think of a museum
as a ‘church for objects’ because, as Boris Groys is pointing out, the museum is the place
where the modern subjectivity is manifested in its most immediate form. The museum
gives us a moment of pure insight, and is the place where the line between art and non-art
is drawn. That is why museums are interesting to investigate further when the question of
new constructions of reality arises. What happens to the “old” competences that Groys
explains would be the definition of a space for free expression as well as a place where
the limits between art and nonart are drawn?:
Die freie Subjektivität kann sich also nur als frei zeigen, weil sie durch das
Museum verliehen, indem wir das Funktionieren des Readymade-Verfahrens
im Museum kennengelernt haben. Die freie Subjektivität kann sich also nur
frei zeigen, weil sie durch das Museum von der Arbeit der Kunstproduktion
entlastet wird, indem das Museum diese Arbeit übernimmt… Das Museum ist
der Ort, an dem der Unterschied zwischen Kunst und Nicht-Kunst hergestellt
wird.2
In fact, institutions like museums produce a free subjectivity, according to Groys. We may
add that the museums are augmenting the idea of free subjectivity into society. This idea
of a free subjectivity is part of the creation of the museum as a free space where the
framework for a social community is being formulated through artistic/cultural
manifestations and artefacts.
In the age of media the function of the modern museum has become central to the effort of
maintaining an illusion of an inner free space, and one construction of reality as the only
true construction:
Es ist also die primäre Funktion des Museums, die illusion eines Freiraumes
ausserhalb seiner Wände zu erzeugen, der als Wirkungsfeld der freien
Subjektivität fungieren kann. s.12
The modern museum traditionally creates a free inner public space which has
consequences for the outer social space. But the free inner public space as a construced
2
Boris Groys, Logik der Samlung, s. 10
reality is in a crisis. There are many reasons for this crisis, which has lurked for a long
time, but the main reason is to be found in the very core of the discussion of augmented
reality in art – that of the paradigmatic change of the traditional public spaces by a media
consciousness.
The challenges of augmented reality to the modern public space could be summed up with
a reference to Jürgen Habermas, who believed that the modern public space was founded
in a literary consciousness. How would this, I am very tempted to ask, change if the public
space is founded in a media consciousness instead?
A reality under construction is a process which is totally depending on temporality. But it is
also a focus: It is a way to be aware of the world and the media, and of the people in it.
Augmented reality - as a reality under construction in contemporary art - is located in a
media conscious practice.
Historically, media consciousness is present on three categories (levels) of practice: an
(old) media conscious practice (further subdivided into early media conscious practice and
late media conscious practice), intermedia conscious practice, and new media conscious
practice. These are not historic periods, but are phases of media consciousness that, for
the most part, are still active today. The diagram below gives an idea of the outline of
media conscious categories.
Three main points arise from the level of practice:
First, that the idea of augmenting reality has a history which is linked – closely, I would
claim – to that of media consciousness. Media consciousness is my investigative term that
I introduce to be able to analyze the paradigmatic change in art that take place in the age
of media.
By Media consciousness I mean the rising sensibility of media in art brought on by the ‘age
of media’ – it is the uses of media, the formalistic changes because of media, and the
questions media arises for/in art. But it is also a completely new way of looking at things –
and using everything as potential artistic material. Media, then, is not only electronic – it is
also language, images, numbers and genres, even. Time, performativity, institutional
critique and experiments across genres are important features in the media conscious
practice. And, perhaps surprisingly, the history of media consciousness encompasses the
entire 20th century – from early avant-garde to new media in the beginning of the 21th
century.3
I will explain further what the categories stand for. But mainly, I would like to investigate
augmented reality as a media conscious practice; using examples taken mainly from the
Scandinavian art scene I will show how augmented reality as a strategy evoke the different
categories are active in a variety of transdisciplinary experiments where reality is under
construction.
Transdisciplinary challenges – new competences and new domains
Writing in 2004 for the 25th anniversary of Ars Electronica in Linz, German art historian and
director of ZKM in Karlsruhe, Peter Weibel, suggests that “a transdisciplinary mapping of
competences is taking place in the fields of art.” He continues:
[…] The redistribution of competence … is precisely the progressive status of
the contemporary avant-garde: […] its protagonists attempt to undertake new
comparisons and redistributions of competence, and to make the transition
from the formal level to the level of practice […] Extension of competence is
the latest phase in the expansion of the concept of art. Art expands from the
object to the practice and its practice expands from its field of work into new
domains...i
A good example would be George Meliés that made the silent movie Long
distance wireless photography in 19113. There is nothing new on the artmedia marked!
Nobody really understood the context of this work until media art was defined.
3
What Peter Weibel is formulating here, is a fundamental framework for understanding
contemporary art practice as something that involves more than aesthetic and formalistic
research. There is much more at stake since the competences of art and art institutions
are being redistributed. In fact, I would claim, that this is the case with any augmentation of
reality: Well known domains of knowledge and art are shattered and sometimes even
dissolved, and it there places new domains will occur. It becomes a question of formulating
qualitatively new phenomena and inventing concepts and strategies to do so, on the fly, so
to speak. Augmented reality, reality under construction, is the destruction and construction
of new domains. In the following I will take a closer look on what those new domains could
be – and how the field of competences within art, organization of knowledge and
institutions are changing.
Art-genres and art-institutions are under construction; they are involved in a process where
the redistribution of competences and new domains for research and knowledge is taking
place. But what does that mean? What, indeed, is being augmented, and how?
The Mediator
The examples that I have used in my investigation of the “mediator” so far are “raw” in the
sense that they only play a part on a partial level in my argumentation. They mainly
represent thoughts, ideas, and strategies that constitute the practices of the artists as
mediators. The examples are not by any means to be seen as in-depth analysis of the
artworks in question, nor do they constitute any elements of an “art history”. For them to be
that would require that the field would be transparent in some kind of totality – or, even
worse, that I would generate the notion that what I am examining is limited to the field of
“art”. That is, that the mediator is existing “outside” everyday experience and “not part” of
the public or political spaces of the reality under construction. I wish to point out how the
mediators are investigating and using the new domains and competences already preset
in the public / political spaces of the new media culture, and which are challenging the
cultural institutions of our society. However, there is no doubt that some of the new
practices and new domains will also demand for new institutions – or something
completely different that we have not yet seen.
That being said, I find that there are some key features of the concept of the mediator that
should be mentioned here. First of all, there are precursors: Wark McKenzie investigates
the “hacker” and “the gamer” as the personas of digital culture – asking the very important
question what kind of critical cultural production would follow after the novelist of the 19 th
century and the filmmaker of the 20th century? Where the hacker according to Wack is the
radical and, dare I say, avant-gardist of digital culture, the gamer is the dreamer of a
‘reality’ with rules that (most probably) will never be.Other important precursors of the
mediator: The “raw” (and the cooked) by Claude Levi-Strauss; the “flaneur” that walks the
pages as well as the real streets of Baudelaires books. The idea of the absolutely modern
artist; even the notion of the “avantgardist”; all these could be seen as mediators of their
time, at some level.
The notion of reactive mediators is taken into serious consideration in the practice of the
Danish mediaartist Mogens Jacobsen, who explicitly “hates the concept ‘interaction’ 4. This
is more than an aesthetic comment – it is a structural remark on the status of an entire
field of practice. Interaction has become a kind of logic-by-default that we use to ‘explain’
New Media. Instead, Mogens Jacobsen introduces the concept of ‘Branching.’ And
‘reactive media’: “Reactive media is branching without story.” Jacobsen’s practice could be
categorized as ‘responsive media’ or ‘reactive media’.
The point for Mogens Jacobsen is that we must never stop looking for the true
epistemology of real-time, pervasive computing and augmented reality.
Audiobar is doing just that.
Audiobar is part of Media Art Platform. The idea with Media Art Platform is to
use New Media technology in experiments where three things should happen:
1) Digital material from the collection of Museum of Contemporary Art
becomes physical present in a body-space.
2) Fugitive and intermedial artforms are distributed to an audience in a direct
and perceptive way through augmented or pervasive technological designs.
3) The audience should be able to have a lot of fun with complex and even
difficult material – the complex and difficult meets Superman.
To achieve these goals, The Museums of Contemporary Art in Roskilde wish
to invite different mediaartists to develop different solutions using New Media
to its very limits – and beyond.
“I hate: 1) A narrative – thinking about navigation in a storyline. 2) The timeline”. Quote from:
Morten Søndergaard: Notes on Mogens Jacobsen, Get Real! Seminar, at KIASMA, Helsinki, 2004.
4
But there are other things to consider. Media conscious art is research. Danish Performing
poet Per Højholt shows this already in the late 60s, and Icelandic media artist Steina
Vasulka was investigating Real-time as a video-art concept in the late 60s and early 70s.
For Vasulka, It was a case of “get something going – elaborately – and then step away.
Let the machines do the job.” At that time it was more like an editing concept, it was a
montage-use.
According to Vasulka, video was considered an inferior media, but it had something that
could be used.
End
The question of how and why augmented reality has to do with the configuration of
contemporary art is really the question of how and why contemporary art define itself
beyond aesthetics - and beyond art, even.
The question of how and why is, in fact, exactly more a question than an interpretation …
is AR an indication of a certain role in or perhaps a particular function of art, making
augmentation a kind of narrative or pictorial artistic strategy? Or is it a contextual – or
maybe even a social – practice that, once again, places the receiver at the center of
attention and the creation of meaning? Does augmented reality instigate qualitatively new
phenomena? Or is it merely a new concept for something as old as art itself: the question
of how art relates to reality?
Once again, we could say that the relation of art and reality is being investigated, and that
augmented reality is just another term (or excuse) for getting that old show on the road under the pretext and inspiration of technology, naturally.
But a reflexitivity is at play in the formation of the term and in the practice of augmented
reality that at least to begin with seems to be new – and I almost already expressed it: it is
not just a concept, not just another term for art trying to grasp reality (because then it
would be reducible to the a discussion of mimesis or representational strategies, which it is
not); it is as much the other way around: reality that is investigating, using and becoming
art. It is a conceptual frame for something which happens – earlier, now, or in a moment. It
is indicative of temporality in art as well as in knowledge.