Jan Koenderink
Gestalts and Pictorial Worlds
Gestalts and Pictorial Worlds
In a well known artistic manifesto Maurice Denis (1890) wrote: “Remember
that a painting – before it is a battle horse, a nude woman, or some anecdote – is
essentially a flat surface covered with colors assembled in a certain order.” Although
you will probably accept this as literally true, many observers find it difficult to
see (experience) this. (See figure 1.) Visual artists in the naturalistic traditions
had academic training in the ability to actually see “colors assembled in a certain
order,” which is necessary to judge the composition of a painting (Pier Paolo
Luderin 1997). They often use tricks like viewing the picture upside down,
squinting, or looking through their eye-lashes (Betty Edwards 1999). In nonnaturalistic art-forms (think of the late Piet Mondrian, of Jackson Pollock, and
so forth) such devices are unnecessary, then the “colors assembled in a certain
order” are all there is to the work.
Fig. 1 “Colors assembled in a certain order” (here merely black and white; a fragment from Frank
Miller’s “Sin City”). Would it matter if you turned the picture upside down?
In naturalistic art most people become immediately aware of the “battle horse,”
or the “nude woman.” (The “anecdote” is not an element of immediate awareness
at all, that Maurice Denis added it is only a historical accident.) Many painters
often feel they apply brush strokes to cheeks or noses, although they understand
very well that they are in the business of brainwashing their prospective
GESTALT THEORY
© 2011(ISSN 0170-057 X)
Vol. 33, No.3/4, 289-324
GESTALT THEORY, Vol. 33, No.3/4
clients through the application of pigments on a planar surface. Edgar Degas
remarks “A picture is something that requires as much knavery, trickery, and deceit
as the perpetration of a crime.”
Here I am mainly interested in “presentations,” that are the momentary content
of visual awareness. Presentations just happen. Even adding “to you” would be
incongruous, because ”seeing happens” is more accurate than “I see.” (Compare
Maurice Merleau-Ponty 1993.) Thus presentations are pre-cognitive. This is not
to say they are irrational, for presentations make plenty of sense. Otherwise we
wouldn’t be here! Perhaps one might call them “proto-rational” (Rupert Riedl
1984) because of their pre-reflective nature.
On the microgenesis of presentations – I will use the term “iconogenesis”
in the remainder of this paper – little research exists (Jason Brown 1996).
Phenomenologically, a “specious moment” (see William James, 1893, 609)
may contain up to about a dozen presentations. As presentations happen, they
disappear. One “beat” of this systolic process takes perhaps a tenth of a second.
Although presentations proper indeed disappear the moment they happen, their
contents often enter cognition, albeit stripped of their qualities, and with their
meanings abstracted and formalized. This then becomes the topic of cognitive
science—the heir of behaviorism, which refuses to deal with presentations proper.
The resulting thoughts (as distinct from presentations) are mere empty shells as
compared to reality (awareness). As a consequence they can be manipulated
voluntarily. They can be “known,” whereas presentations cannot, this being the
reason why many scientists prefer thoughts, and go as far as to deny the very
reality of presentations, bizarre as it may appear (Daniel Dennett 1991). Strange
indeed, for to deny presentations is akin to denying reality! The “qualities” and
“meanings” in presentations cannot be preserved in reflective thought, where a
mere proxy of reality is artificially split in mind, body and environment.
An in-depth study of psychical disorders leads to the notion that presentations
are the final stage (an “end product”, like a rigidified crust) of an unfolding
process. This “evolution” is analogous to the evolution of species, but occurs on
a tremendously shorter time-scale. It involves the diversification and pruning
of numerous hallucinatory threads (Brown 1996). Such threads start out as
undifferentiated feelings, or emotions, and pass through dreamlike stages,
involving reiterating fragments of the history of the observer. They are constrained
by the “colors assembled in a certain order.” Consequently, viewing a picture of
a “battle horse” will rarely lead to the awareness of a “nude woman,” although
such is by no means ruled out. The constraint ensures that the presentations are
“about” the picture. Presentations within a specious moment tend to be similar,
though differences can frequently be noticed. Think of Necker-cube reversals
(Louis Albert Necker 1832) as a simple example.
290
Koenderink, Gestalts and Pictorial Worlds
Notice that iconogenesis proceeds by progressive diversification, rather than by
the accumulation, and assemblage of parts. Meaning can be attached to parts
because the whole acts as context, whereas the meaning of a whole cannot derive
from its parts. Likewise, in a painting the brush strokes, and so forth, derive
meaning from the painting as a whole, not vice versa.
One may describe iconogenesis formally as the development of meaning through
creative guessing, kept in check by probing for falsification (Jan Koenderink
2011). Probing is an intentional (Franz Brentano 1874) action, much like asking
a question. But notice that questions imply possible answers, that is to say, any
meaning is in the question, rather than in the answer. This is especially true
when asking questions to nature, rather than to a fellow human being. Different
from a person, nature does not return meaningful answers, it merely yields
(meaningless) structures, say Arthur Eddington’s (1929) “pointer readings.”
The optical apparatus of the observer yields sampled optical structure, which
is intrinsically meaningless (George Berkeley 1709; Eddington 1929). When
probing meets resistance, this yields information about the world that is
meaningful in the context of the intention of the probing. This might be seen
as an instance of learning by mistakes. Such “controlled hallucination” is an
information generating process, perhaps the only such process we know. That
science progresses by exactly (at least formally) the same method is rarely
acknowledged. However, “meaning” in awareness is categorically different from
“meaning” in science. It is neither subjective, nor objective, as the self does not
exist in awareness. Although there can be no mind without awareness, awareness
does not depend on a mind.
This was clearly understood by Erwin Schrödinger (1944), who equated the
encounter of a resistance to intentional probing with a “spark of awareness,”
a micro-enlightenment as it were. Here I am interested in the initial “creative
guesses.” They appear similar to the “releasers” of ethology, and quite possibly
lie at the basis of the conventional “Laws of Gestalt” (Wolfgang Metzger 1992).
Donald Hoffman’s notion of “perception as a user interface” echoes the same
ideas (Hoffman 2008; Hoffmann 2009). This user interface notion fits very
well with ideas from ethology (Jakob von Uexküll 1909; Niko Tinbergen 1951;
Konrad Lorenz 1973; Rupert Riedl 1984).
Visual artists have developed a sophisticated praxis aimed at the triggering of
pictorial worlds in contemporary, generic observers, their prospective clients.
In experimental psychology these are often denoted “naïve subjects,” actually
they are mostly connoiseurs. In modern Western art this has developed in
synchrony with insights from psychology, the interactions often being weak,
but bidirectional (Marianne Teuber 1973; Teuber 1974; Teuber 1976; Rudolf
Arnheim 1943; Arnheim 1966; Arnheim 1974; Arnheim 1986; Hoffman, 1998).
Artists have typically concentrated on more complicated cases – the works by
291
GESTALT THEORY, Vol. 33, No.3/4
which they have to make a living – whereas scientists tend to concentrate on the
simplest possible cases–the aim being to get at the essential “roots” of the matter.
Only recently Baingio Pinna & Liliana Albertazzi (2011) extended experimental
phenomenology of visual patterns to rather more complicated cases. The
approaches have been largely complementary.
In this paper I try to approach the more complicated – hence intrinsically more
interesting from an applied, or cultural perspective – case from a scientific
viewpoint. The culturally important incentive is the topic of “visual literacy,”
(Maria Avgerinou & John Ericson 1997) which has numerous implications in
education, communication, and so forth. The scientific incentive is the hope that
something like a “grammar of vision” might be construed as more than a mere
elusive fashion word.
Pictorial Worlds
Style and Gist
Whoever is conversant with the visual arts will – at first glance–be aware of the
cultural background, the style and the approximate date of a picture. It is often
difficult to verbally “defend” this immediate, pre-cognitive conviction. This
takes reflection, and is evidently something that happens after the fact of the
immediate awareness.
The “style” of a painting is a Gestalt. This is important in art appreciation. One
may equally enjoy Western medieval, renaissance, baroque, impressionist, etc.,
paintings, as well as Japanese prints, Roman mosaics, and so forth, but only if
one intuitively “places” such artifacts. Each can be enjoyed as an achievement in
its proper context, even though it might count as “primitive,” “ugly,” “weird,”
“incomprehensible,” …, in another (perhaps one’s current cultural framework)
context. Such Gestalts are evidently composite, even though they appear as single,
indivisible entities, e.g., like “pointillist seascape.”
Something similar applies to “gist.” Without any parsing of the image one
feels to be confronted with a landscape, marine, still life, portrait, figure or
drapery study, trompe-l’œil, and so forth (Aude Oliva & Michelle Greene 2009;
Greene & Oliva 2001). Of course, this interacts with the style, as some styles
preferentially include certain gists, or are oblivious of them. The awareness of
the gist is also immediate, and pre-reflective. Gist too, is evidently complex, yet
appears as single, indivisible entity, e.g., like “flower still life.”
That both style and gist are Gestalts is illustrated by the fact that they can easily
be abstracted and captured with a few lines in a cartoon drawing, or rough brush
strokes in the “background” of a painting. Such renderings are not necessarily
recognizable in the usual sense. (E.g., Suzanne Unrein, 2011, paints abstract
292
Koenderink, Gestalts and Pictorial Worlds
works such as “a battle scene in the baroque style”. One immediately recognizes
these as such.) Style and gist exist more or less independently of the nominal
“content,” like the anecdote, battle horse, or nude woman.
The Shape of Space
“Pictorial space” is a complicated notion that has various dimensions to it
(Koenderink et al, 2011; Andrea van Doorn et al, 2011b; Johan Wagemans et al,
2011). For instance, it has two- as well three-dimensional connotations, either
one important. In this paper I mainly consider the latter, though it is really
impossible to ignore the former. The latter case involves the notion of “depth.”
Two of the three dimensions may be identified with the picture plane, or the
visual field, the third one is depth.
“Depth” is a feeling of remoteness. One is perhaps ready to say “remoteness from
the self,” but the more basic feeling occurs on a level where the “self” does not
figure. In naturalistic paintings “depth” may be rather articulate, but in other
cases it may be only weakly developed.
Depth is not present in the picture plane. Thus pictorial objects with very
different depths, may actually touch, or overlap as planar structural elements.
This is an important compositional device, one reason why the two- and threedimensional structures continually interact, and cannot really be separated. In
Adolf von Hildebrand’s (1901) terms, different depth layers “join hands” by way
of the two-dimensional arrangement. (Example in figure 2.)
Fig. 2 Although there is depth, there is not really any “space” here. One sees a complicated object,
a “heap” of blocks say. The heap results from the two-dimensional overlap, although the blocks
are seen in different depth layers they “cohere” because they touch in the picture plane. Adding a
picture frame would add an empty, enveloping space, this is discussed in the next section.
293
GESTALT THEORY, Vol. 33, No.3/4
In paintings or drawings the feeling of remoteness often takes a certain “shape,”
different in landscapes, still lifes, portraits, and so forth. Such “shapes of space”
are in immediate awareness, and are hard to verbalize, though often decisive in
the esthetic impression of the painting. Shapes of space are intimately connected
with both style and gist, they co-occur and co-determine each other, although
there are no linear dependencies. The shape of space is an important Gestalt
in its own right. It too, is evidently complex, yet appears as a single, indivisible
entity, like “an open space.” (Examples in figure 3.)
Fig. 3 Pierre Bonnard, two plates from Daphnis et Cloé (1902). There is plenty of space here, yet
the two examples have a very different “shape of space.” Notice how the objects “dissolve in the
space.” Both spaces have an obvious center, the left one is “closed,” the right one “open.”
The shape of space is co-determined by the objects that are in pictorial space,
and may be almost fully determined by them. But this is by no means necessary.
The shape of space may be determined by some general spatial framework (see
below), or by the mere (pictorial) light field. In the case where the artist uses light
and dark, the shape of space, and the light field tend to be intimately entangled,
and appear as a single Gestalt. (Example in figure 4.)
The most primitive space due to light is perhaps the region surrounding the
firepit of a campsite, and was familiar to early homo. It creates a relatively safe
region around it because it discourages animal predators. Moreover, it relates to
comfort, and food. It intuitively feels like “hearth and home.” One automatically
experiences this in pictures.
294
Koenderink, Gestalts and Pictorial Worlds
Fig. 4 John Constable’s View on the Stour: Dedham church in the distance illustrates the fusion of
the shape of space, and the shape of light.
The Picture Frame
In the bulk of conventional visual art the picture frame is given. Even in cases
where the picture frame is apparently lacking (like a small drawing in the middle
of a large sheet), there is almost always a frame-like context of some kind.
Occasionally (as in Japanese fan-paintings, Western lunette-paintings, and so
forth) the frame may have a non-rectangular shape.
The frame itself is an important Gestalt that is very much part of the work. It
is used and respected by all professional visual artists. Often it has a decisive
influence on the composition. (Examples in figure 5.)
295
GESTALT THEORY, Vol. 33, No.3/4
Fig. 5 Left: Alfonse Maria Mucha, drawing for a menu card (1897). Right: Mucha, Drawing from
the Figures Décoratives (1905). Notice how Mucha adapts the drawing to the frame.
There exists quite a volume of writings on the influence of the frame, and how
to take advantage of it in composition (Jack Hamm 1988; Walter Crane 1900).
Such insights are potentially useful in attempts to set up a “grammar” of pictorial
worlds (see below). It allows one to order the nexus of juxtaposed and nested
Gestalts in a spatial framework.
The Ground Plane
One very common entity that is closely related with the shape of space is the
(pictorial) ground-plane (James Gibson 1950; Hamm 1988). It is implicit in
many pictures, and becomes explicit in sophisticated paintings of landscapes,
and interiors. The ground-plane and the shape of space co-determine each other,
either of them may become dominant. The ground-plane is an important Gestalt
in its own right. It is interesting because it is rarely drawn or painted as such.
Usually it is only hinted at, but even so it tends to be a major determinant of
pictorial space. The history of landscape painting in Western art is for a large
part the history of the ground-plane (Kenneth Clarke 1949).
The ground-plane was actually drawn in the art of the early renaissance (Leon
Battista Alberti 1435). This was the legendary pavimento of linear perspective.
It may yield the impression of infinite extent, even when drawn all by itself. It
can be used to “place” objects in pictorial space in a rather mechanical, though
visually compelling manner.
Historically, the explicit pavimento soon went out of fashion with the artists.
In later periods one mainly encounters it in interior scenes (tiled floors, floor
boards), where it appears natural if sufficiently deemphasized. The pavimento
296
Koenderink, Gestalts and Pictorial Worlds
appears in other cultures too, although it is only in Western art that it reaches
out towards infinity (the horizon). (Examples in figures 6.)
Fig. 6 At left the renaissance pavimento, running up to the horizon. At right a “pavimento” as
found in Japanese prints. (It would indicate tatami mats). It suggests only a limited area, but
serves very well as a ground-plane.
Early post-renaissance landscape art struggled with the ground-plane in the
absence of a pavimento. This led to a system of foreground, middle-ground,
background depictions, with “gaps” in between, usually camouflaged through
“artistic fig leaves” (Erwin Panofski 1925) such as occluding hill tops, or water. The
advantage was that the depth layers could easily be given distinct remotenesses,
often by means of a color-code brown (foreground) — green (middle ground) —
blue (background). This yields ground-planes of limited extent, but a clear shape
of space.
In most art the ground-plane is not drawn as such, the main exception being a
horizontal line on which various objects are placed, like in children’s drawings.
This may be done in a less obvious manner by drawing objects on the same
height in the picture plane, then the horizontal line–while absent–will magically
do its job. (Various examples in figure 7.)
Fig. 7 In cave art (top left) the ground-plane is absent. An explicit line (top right) nails everything
to the ground. In more sophisticated drawings (bottom, by Felix Valloton) the ground-line is
implied, but notice how clearly it is “seen!”.
In early art one gives each object a small segment of horizontal “ground-line.”
This evolved into a practice of drawing short pieces of cast shadow, with the same
effect, but perhaps appearing more “naturalistic.” Transitional cases are seen in
297
GESTALT THEORY, Vol. 33, No.3/4
the floor mosaics of the Piazza Armerina, a Sicilian roman villa, made by North
African artists in the early fourth century CE. Splitting the ground line enables
one to place objects at different heights and still have them “sit on the ground.”
The next step is to use higher in the picture plane as more remote. This again
evolves into an implicit rendering of the ground-plane through the placement of
objects. Examples are Persian miniatures. A further evolution is to use familiar
size in addition to height. (A sophisticated example is shown in figure 8.)
Fig. 8 A sophisticated implied, and inclined ground-plane by Felix Valloton.
Fig. 9 The surface of the water (the “groundplane”) is defined by the elliptical ripples in The
Bath by Felix Valloton. The intersections of the
bodies with the water surface are also elliptical, and
serve to strengthen the plane. They also repeat the
elliptical ripples.
298
Koenderink, Gestalts and Pictorial Worlds
When no convenient objects are available, like in barren landscapes, one draws
elements of the ground such as stones or weeds. The ground-plane is emphasized
by giving them easily recognizable shapes, for instance foreshortened circles
drawn as ellipses. Waterlily leaves thus come to define the water surface which is
hard to paint by itself. (See figure 9.)
Combinations of such methods enabled late nineteenth century Western artists to
suggest infinitely extended landscapes that immediately enter visual awareness.
An analysis usually shows how the suggestion of the ground carries the shape of
space. (Examples in figures 10 and 11.)
Fig. 10 Using the implied ground-plane, occlusion and size cues, it is easy enough to draw a
“deep space” depiction.
Fig. 11 In this picture of a school bench by Felix Valloton the size cue works against the depth
due to the implied ground-plane, yielding a comic effect. This is funny in immediate awareness,
though hardly of noticeable interest in reflective thought.
299
GESTALT THEORY, Vol. 33, No.3/4
Pictorial Objects
Pictorial objects are structures that cohere in a single pictorial entity, but are
parts of the picture as a whole. They often — though by no means necessarily —
can be recognized and named.
This latter point is largely irrelevant to my quest. Visual artists are conventionally
advised to avoid recognition, and naming, because it hinders picture making due
to the introduction of cognitive elements. Such cognitive elements have nothing
to do with picture-making proper. Only Sunday painters paint noses, lips, or
cheeks, and this shows (detrimentally I’m afraid) in their results. A “grammar of
vision” is certain to be very different from a taxonomy of physical objects (Sed
Jacobs 1988).
Once this is thoroughly understood, it is harmless to mention a “nose” as a shortcut to indicate a region on the picture surface. In this paper I freely do so. Just
don’t forget that it is a patch of paint, a shape in visual awareness, and so forth,
but definitely not a nose. Moreover, how would you delimit a “nose,” even in
physical space?
One way to counteract the force of recognition is to ask yourself “what else is it?”
In photography this was Alfred Stieglitz’s (1864–1946) notion of “equivalents,”
that has become the mark of careers like that of Minor White (1908–1976). (See
Alfred Stieglitz 1923; Minor White 1923.)
Visual awareness is composed of the rigidified outer crust of the evolving content
of iconogenesis (Brown 1996). Occasionally one believes they catch glimpses of
such an evolution. This is especially the case when the constraint posed by the
“colors assembled in a certain order” is weak. A well known example is Leonardo
da Vinci’s (1835) remark on what can be seen in a random pattern:
“… look into these stains on walls, the ashes from the fire, the clouds, the mud, or
other similar places. If these are well contemplated, you will find fantastic inventions
that awaken the genius of the painter to new inventions, such as compositions of
battles, animals, and men, as well as diverse compositions of landscapes, and
monstrous things, as devils and the like …”
Some modern artists have used similar insights to produce remarkable works
(Robert Pepperel 2006). In their paintings one believes they experience the fluidity
of the iconogenetic process through one’s fluctuating stream of presentations.
Pictorial objects, irrespective of whether they are recognized as something or not,
tend to have a variety of qualities, like shape, material properties, and so forth.
They also tend to have histories and futures. Often they causally act on other
pictorial objects.
If you recognize a “dented coke can,” you became aware of an object that was
300
Koenderink, Gestalts and Pictorial Worlds
“once” symmetrical, whereas it has “now” a perhaps more interesting shape
(Michael Leyton 1992; Marcel Hendrickx & Wagemans 1999). You are likely
to see this even if you never saw a coke can before in your life. It also looks
like an artificial, man-made object. You ascribe the “dent” to an unfortunate
event in the object’s history, it will never occur to you (at least not in immediate
awareness) that it might be a pop-art object, manufactured in a dented-like shape
at considerable cost and effort.
Many pictorial objects have obvious futures (Ernst Gombrich 1961). A pointed
object tends to “move” in the direction of the point, and so forth. The famous
Discobolus of Myron (circa 460-450 BCE) is the familiar example. (N.B., works
mentioned in the text, but not illustrated, may immediately be found through Google
search.) (See figure 12 for a striking example.)
Fig. 12 Pictorial objects have pasts and futures. In this painting by Dali time appears to run very
fast, yet nothing changes. The presentation catches the “eternal moment now.”
It is surprisingly easy to draw abstract shapes that are “seen” to causally interact,
e.g., to inflict harm on each other. Compelling, and often surprising examples
are given by Pinna (2010). This is the reason why even simple drawings in comic
books are so immediately (visually!) convincing. (Figure 13 shows a Picasso
drawing were various interactions between pictorial objects are visually evident.)
301
GESTALT THEORY, Vol. 33, No.3/4
Fig. 13 In this “dance” by Pablo Picasso the two persons causally interact.
Pictorial objects thus always come with a past and a future. In Gottfried Wilhelm
Leibniz’s (1714) terms “as every present state of a simple substance is naturally a
consequence of its preceding state, so its present is pregnant with its future”. The past
and the future derive from the entire life history of the observer. Time appears
as a context in which the objects are embedded. Although presentations are only
the “moment now,” they are apparently enveloped in time. This temporality is
an important, even defining quality of every pictorial object. (Compare Lord
Shaftesbury’s (1714) notion of “anticipation and repeal.”) The visual artist is
familiar with the topic through “gesture drawing” exercises. (Example in figure
14.) When drawing full figures one will likewise start with a single “line of action,”
this has to be perfectly right before one even thinks of drawing more. The line of
action alone catches the expressive tension of the body. It is a basic Gestalt.
Fig. 14 “Gesture drawings” of hands. Fragments from a page by Jeff Mellem (2009).
302
Koenderink, Gestalts and Pictorial Worlds
The temporal envelope of things has been represented by artists from the earliest
times on, they are seen in early cave drawings, in the classical Greek statues,
and throughout Western art. In works of the futurists (like Umberto Boccioni’s
“unique forms of continuity in space” (1913) in the Museum of Modern Art,
New York) it appears to be the major theme. In a 1914 manifesto Boccioni
wrote “While the impressionists make a table to give one particular moment and
subordinate the life of the table to its resemblance to this moment, we synthesize every
moment (time, place, form, color-tone) and thus build the table.”
(In figure 15 I
show a futurist painting by Giacomo Balla.)
Fig. 15 Giacomo Balla’s Dynamism of a Dog on a Leash (1912). The dog is enveloped in spacetime.
Pictorial objects and elements of the “colors assembled in a certain order” cannot be
mapped on each other in a one-to-one fashion. This is evident from the familiar
fact that the same elements may be associated with distinct objects. Jastrow’s
(1899) duck-rabbit is a famous example (Ilse Verstijnen and Wagemans, 2004).
Usually presentations will contain one “interpretation,” but occasionally one
feels they are aware of simultaneous, mutually exclusive images. Examples
are the well known bunch of grapes and the horse, or the bust of Voltaire and
visitors at a slave market, of Salvador Dali. Looking over early Picasso drawings,
303
GESTALT THEORY, Vol. 33, No.3/4
one feels that it might well be possible to draw a square circle! To paint objects
like a golden mountain is trivial, and it is easy enough to paint a white thing
that is black (or rather, something that is white and black simultaneously, see
Koenderink & Andrea van Doorn 2000).The presentations apparently reach out
into “Meinong’s jungle” (Alexius Meinong 1899; Richard Routley 1982). (An
example is shown in figure 16.)
Fig. 16 Left, a decorative border from Alfonse Maria Mucha’s Ilsée (1897). Right, Landscape with
cliffs by Edgar Degas. In either case the earth is personified as a woman. Here we are in danger
of entering Meinong’s jungle. This is even trickier in examples that are less obvious than this one,
of course.
The histories, futures, and causal actions of pictorial objects, like the pictorial
objects themselves, are Gestalts. They exist in addition to their parts, if they can
be said to have any. Once the whole appears, the parts sink into oblivion. They are
not independent though. In a picture anything is synchronized with everything
else. A pictorial world is like a Leibnizian universe of monads (Gottfried Leibniz
1714), or — perhaps more apt, because more dynamic — like an instance of
Indra’s net (Francis Cook 1977).
Pictorial Shape
Pictorial shape is another complicated notion, with many different aspects to it.
In this paper I mean “shape in depth” when I simply say “shape,” whereas I use
“form” to denote a shape (the word used in a sloppy sense at this point) in the
picture plane or visual field. Objects have shapes, and so have patches of surfaces
of objects. Such surfaces may be called “reliefs” (Adolf Hildebrand 1901), and
the shape of small patches of relief is conventionally denoted “curvature.”
Surface curvature is a well developed concept in geometry (Carl Friedrich Gauss
1827; Koenderink 1990). Surfaces are curved independently in two mutually
perpendicular directions. If the two curvatures are of the same sense one has
either a convexity, or a concavity, if of opposite sense one has a saddle shape. If one
curvature vanishes the patch is cylindrical, if both vanish it is flat. Generically,
304
Koenderink, Gestalts and Pictorial Worlds
surfaces are locally either convex, concave, or saddle shaped. Cylindrical shapes
occur only as transitional cases, and flats don’t occur at all. Of course this
(perhaps perversely) fails to apply to the man-made world, where materials are
often forced into non-generic shapes, though — obviously — not to last.
Such a formal, geometrical taxonomy does not seem to apply to visual awareness
though. Leon Battista Alberti (1435) drew up the first taxonomy of experienced
shapes in the early fifteenth century, and omitted the saddle shapes. Despite
great, and continued interest in Alberti’s work no one noticed the omission until
recently (Koenderink & van Doorn 1992). (The taxonomies are illustrated in
figure 17.)
Fig. 17 At top the surface shape taxonomy of Gauss, a convexity, a saddle and a concavity.
Only generic surface types are recognized. At bottom the surface shape taxonomy of Alberti,
a convexity, a concavity, a convex cylinder, a concave cylinder and a flat. Alberti recognizes
degeneracies like cylinders or flats as regular cases, and fails to include the saddle shapes. The
(deficient from the geometers point of view) taxonomy of Alberti appears to represent the actual
Gestalts recognized by the human observer. It is psychology, rather than geometry.
Curvature is often suggested through shading in both drawing and painting.
A standard stimulus in experimental psychology has been a circular disk filled
with a linear gradient (Vilayanur Ramachandran 1988). From an analysis of the
physics (Koenderink & van Doorn 2003) one finds that this might depict any
possible curvature (convexity, concavity, saddle, or cylinder) except planarity. The
literature only mentions convexity and concavity as visual experiences though
(Ramachandran 1988; Steve Palmer 1999; van Doorn et al, 2011a; Wagemans
et al, 2010). Apparently no one even noticed the absence of saddle judgments.
Thus convexities and concavities appear to be the local Gestalts related to reliefs.
Reliefs (Adolf Hildebrand 1901) may be very complicated, think of the surface
of the human body. They are seen as conglomerates of simpler components,
typically convexities or (almost) cylinders. In sculpture classes one is advised to
mind the convexities, anything else taking care of itself. Leonard Rogers (1969,
60) has:
305
GESTALT THEORY, Vol. 33, No.3/4
“Sculpture students modeling from the living model used to be told to take care of
the positive forms and leave the negative ones to take care of themselves. They would
arrive at the hollow of an armpit or a navel or the channel of the spine by building up
the convex shapes that surround them.”
(See figure 18 for an example showing an academic nude treated like a “bunch
of grapes.”)
Fig. 18 William Rimmer Full muscular detail. The academic nude illustrates the possible
complexities of pictorial reliefs. The surface detail is made up of a precisely ordered arrangement
of ovoid bulges.
In drawing one often indicates concavities with dark, convexities with (suitably
shaded) light (Jacobs 1988). The convexities are often grouped into ranges of
hills (William Rimmer 1877).
There is a natural tendency to divide the shape into simple parts. Rogers (1969,
62) has:
“The transitions between the volumes of the so-called primitive wood sculpture of
Oceania and Africa are almost without exception abrupt and each major part of the
sculpture is carved as a separate, self-contained, and clearly delineated volume.”
And, Rogers (1969, 63)
“Perhaps the most extreme contrast to the abrupt transitions of African art is offered
by the work of such twentieth-century sculptors as Arp and Moore. In many of Moore’s
works there are no abrupt transitions at all; each form blends with its neighbours to
provide a rhythmic continuity of surface and contour.”
306
Koenderink, Gestalts and Pictorial Worlds
(See examples in figures 19, 20 and 21.) When sharp transitions are used one may
skip saddle-like reliefs completely.
Fig. 19 Some volumetric shapes indicated mainly through convex contours and T-junctions.
Using such simple shapes suffices to draw essentially anything! Such blobby volumes suggest
depth, twists, “growing of appendages,” bending, and so forth. They are easily invented on the
fly. More complicated shapes can be build up by juxtaposition as in figure 21, although one has
to mind Roger de Piles principle of the bunch of grapes (figure 28).
Fig. 20 “Cubical” volumes. Such simple “chunks of goo” also suffice to draw virtually anything
once you get the hang of it! They can be easily combined to form larger units. The chunks are very
flexible, as they can easily be bended or twisted.
Fig. 21 Drawing by Dali. The gist is a “nude woman,” sub-Gestalts are ground-plane, and torso,
sub-Gestalts of the torso are ovoid shapes, and marks. Some ovoids or marks are recognized
as anatomical features. The pen-strokes serve a variety of different purposes (shading, contour,
landmark, …), and some groups are Gestalts in their own right. Although the shape is a composite,
it is equally a whole. Notice that this shape was drawn using the building blocks presented in
figure 19.
307
GESTALT THEORY, Vol. 33, No.3/4
Reliefs may overturn in depth, and so come to a natural conclusion. The surface
beyond the boundary belongs to the backside, and is not depicted. René Magritte
plays on this in his painting “The Schoolmaster” (of 1954), showing the back of
a man’s head: the face is at the backside, and of course invisible, because not
even painted! These natural boundaries are known as ”contours.” They appear in
awareness as transitions from front to back, thus the surface is “seen” to carry on
beyond the contour, even though not represented.
It also happens that one object apparently occludes another. At the occlusion
boundary one has two reliefs, one on each side. For one relief the boundary is a
contour, for the other not: the surface is “seen” to extend beyond the occlusion
boundary, not being rendered because of the surface in front of it. The generic
cases of interest are the “T-junctions,” and the “ending contours.” These are
Gestalts in their own right (Koenderink 1990). (See examples in figure 22.)
Fig. 22 Examples of contour drawings by Pablo Picasso. In the nude en dos one finds many
examples of contours, ending contours, and T-junctions. In the nude at right one sees the power
of leaving the completion of the contour to the observer. Notice that the marks at the belly or the
breasts would be meaningless when presented out of context. They are defined by the drawing as
a whole, just as they themselves serve to define the drawing.
Many pictorial objects are of finite size, then it makes sense to talk of their “global
shape.” The first thing in presentations tends to be the form of their silhouettes.
Painters take care that the forms of important objects are easy to “read.” Since
silhouettes are “flat,” it is important to suggest various gradients of depth. The
conventional tool is the modulation of the contour, minding the “lost and found
edge qualities” (Jacobs 1988). (See example in figure 23.) The fact that pictorial
objects are both two-dimensional, and volumetric marks them as very special. The
emphasis may shift between these two aspects. A generally useful description of
such Gestalts is still sorely lacking. Artists simply rely on their good horse sense.
308
Koenderink, Gestalts and Pictorial Worlds
Fig. 23 A study by Georges Seurat (1884-5)
for his monumental painting the Sunday at the
Grande Jatte. Notice the sophisticated use of
hard and soft contours.
In practice shape almost always interacts with shading. The “shape” of things is
almost as much a matter of geometry as of photometry. Shapes like faces and
torsos yield typical shading patterns that are read almost as “bar codes,” and are
obviously Gestalts in their own right. (See examples in figures 24 and 25.)
Fig. 24 In Jacomo da Pontormo’s Santa Cecilia the face is dominated by dark shadow pools. Such
dark blotches generically occur over and over again, one “reads” them easily. In structures as the
face they occur in fixed patters, for instance, the eye sockets will be dark, so will the upper lip just
below the nose, and the upper chin just below the lower lip. Such patterns read like “bar codes,”
even from a distance, or in rough sketches. They are important Gestalts in their own right.
309
GESTALT THEORY, Vol. 33, No.3/4
Fig. 25 These schematic shading patterns by Jack Hamm (1963) illustrate basic Gestalt properties
of the standing nude figure. Notice once again that the light-dark pattern of the body reads like
a familiar “bar code.” These are strong Gestalts and enable even coarse sketches to work at the
first glance.
Material Properties
Pictorial objects can be matte or glossy, hard or soft “to the touch”(in vision!),
and so forth. They are “seen” to possess qualities that are like material properties.
These “material properties” do not have any immediate relation to the material
properties of physics though. These are no conductors and dielectrics in pictorial
space, whereas this is a basic distinction for physical solids. Physics has no chapter
on “white stuff,” whereas this is an important quality of pictorial materials. The
class of “white things” includes such objects as paper, snow, milk, clouds, foam
on beer, polar bears, bread, and so forth, a gamut that the physicists do not know
how to handle as a homogeneous class. Thus the “material properties” of pictorial
objects should be studied for their own sake.
A quality like “golden” is an example of a complex Gestalt. One doesn’t use “gold
paint!” Instead, “gold” is painted as a texture of white, black, and yellow paints.
It is a spatially compound entity that requires spatial and luminous envelopes as
context in order to “work”.
The material qualities depend upon scale, thus, in paintings, on distance. For
instance, treetops have to be painted quite differently according to distance. A
thorough discussion of the topic of foliage in Western art would fill a voluminous
tome. “Foliage” is again a Gestalt. (Figure 26 has some examples of “visual stuff”).
310
Koenderink, Gestalts and Pictorial Worlds
Fig. 26 A single shape and four different “materializations” by Jack Hamm (1988). The overall
shape is defined through contour and shading, which is the same in all four instances. It is a
Gestalt in its own right. The material properties are due to some detailing. The major tool here is
the treatment of the contour. The treatment of hatching in the interior of the contours also serves
to define the material, but is secondary. Notice how the stone is set firmly on the ground, whereas
the cloud hovers in the air, an effect that is arrived at through the suggestion of the ground-plane.
Here we have a number of interlocking Gestalts, shape, material, lightfield, space.
Hermann von Helmholtz (1977) has:
“… objects … in space seem to us clothed with the qualities of our sensations. …
these qualities of sensation belong to our nervous system alone and do not at all
reach beyond into external space. … Yet … the appearance does not end, because
this appearance is, in fact, the original truth”
.
And so it is, the physical properties being derived truths.
The material properties “live” on the substrate of color, texture, shading, the light
field, shape, and so forth. At the same time, these entities in turn depend upon
the material properties. This makes the picture surface (the “colors arranged in a
certain order”) into a sensitive entity, like a vibrating spider’s web, where any local
change influences the whole. (Figure 27 has a striking example of such global
interactions).
311
GESTALT THEORY, Vol. 33, No.3/4
Fig. 27 In this example by Harold Speed (1917) only the hairline is changed. Notice how this
strongly influences the shape of the face. Anything depends upon everything else, the picture is
a monadology.
Pictorial Grammar
Is it possible to flesh out the notion of a “pictorial grammar”? The concept of a
”grammar” derives from the linguistics of natural languages (Archibald Sayce
1911), and was extended over the definition of formal (algorithmic) languages
(Michael Harrison 1978). One means a syntactical account of a system of serial
communication based on discrete structural elements.
Since pictures are not in serial form, nor made up of well defined, discrete
elements, the conventional notion of a “grammar” (in the Chomskian sense (Steve
Pinker 1994)) certainly fails to apply as such. Moreover, it is not particularly
interesting to give a syntactical account of pictures as “colors assembled in a
certain order,” anyway. This is because the pictorial world seen in the picture
implicates monadologies that need not have any relation to “structure” in terms
of classical information theory (Claude Shannon 1948). That is to say, syntax
of complexes in awareness necessarily depends upon semantics. This is the case
because nothing in presentations is mere structure, all is quality, and meaning. A
pictorial grammar would have to deal with the syntactical structure of a semantic
complex.
A syntactical account of the structure of pictorial worlds would perhaps be
somewhat similar to a mereology. (Stanisłav Leśniewski 1992). A pictorial world
is a hierarchy, or a simultaneous order of hierarchies, of Gestalt-like entities. The
part-whole relation in the hierarchy might cause some concern, so I’ll discuss
that issue first.
What does it mean to have a (super-)Gestalt that has (sub-)Gestalts as its proper
parts? The question as posed this way makes little sense, because it places the
former use of “Gestalt” on the same level as the latter use of “Gestalt”. This is
mistaken, hence the prefixes “super-”, and “sub-”. A visual Gestalt often exists
only because of a context, it also exists because of some constellation of figural
312
Koenderink, Gestalts and Pictorial Worlds
elements. Such figural elements are in themselves only that, they only make sense
in being a structural part of the Gestalt. The levels co-determine each other. A
figural element might well be a Gestalt in its own right, in a view that is too
myopic to see the super-Gestalt. They may derive their meaning from the superGestalt in so far as it serves as a context.
The hierarchy of Gestalts necessarily implies a hierarchy of viewpoints. For any
given viewpoint only certain Gestalts “exist,” ones above may serve as a (perhaps
vague) context, ones below as mere structural elements. The classical reference
is the treatise by Roger de Piles (1708), the famous example of the “bunch of
grapes” echoes throughout academic art theory. The “bunch” is not a collection
of grapes, but is a single entity. (See figure 28.)
Fig. 28 Roger de Piles (1708) “bunch of grapes” illustrates how a “bunch” should be a (super-)
Gestalt, the individual grapes (sub-)Gestalts.
Crude, but useful examples of two-tiered hierarchies of Gestalts are the well
known paintings made by Guiseppe Arcimboldo (1527-1593) at the court of
Rudolf II at Prague. A famous instance is the Portrait of Rudolf II as Vertumnus.
At the sub-level one sees an odd collection of fruits and vegetables, whereas,
at the super-level the portrait of Rudof II “magically” appears. (See Thomas
DaCosta Kaufmann 2010).
Of course modern examples abound. In Saul Steinberg’s Santa Claus (figure
29) the first things to appear are probably Santa Claus and a Christmas tree,
then numerous, richly decorated branches appear, then a variety of birds and
ornaments, finally the numerous decorative scribbles that make up the latter hold
your fascination. In this figure one succeeds in enjoying a number of levels in a
single specious presence.
313
GESTALT THEORY, Vol. 33, No.3/4
Fig. 29 Santa Claus by Saul Steinberg. Santa Claus and the Christmas tree are fully merged and
both are seen as complete at the same time. Yet they “share” many substructures. With some
effort it is possible to “see” only the man or only the tree. When you peruse the detailing both
blend in the background.
There is no reason why Gestalts should correspond to simple spatial regions,
in many cases one encounters unexpected, and surprising “mereological sums.”
Examples abound in the Gestalts of conventional compositional arrangements.
I mention two well known cases. Consider the triangle of Raphael’s Madonna of
the meadows (1506) at Vienna, an odd mereological sum composed of Our Lady’s
head, the Christ Child, the Infant Saint John, and Our Lady’s foot. The triangle
pops up in awareness because of the symmetry of a configuration of the major
centers of saliency. Another striking example is the circle with empty center in
Jacopo Pontormo’s Deposition (1528) at the Capponi Chapel of the church of
Santa Felicita in Florence. The circle is elicited by the remarkable sum of half a
dozen holy persons and the languid body of Christ. Such Gestalts were purposely
constructed by the artists. They work immediately to the sense, irrespective of
whether one knows how to “read” the work in the art historical meaning of the
term. These compositions could easily be emulated in non-objective works.
Of course one has not necessarily strict hierarchies, but typically only partial
orders (Bernd Schröder 2003). A particularly successful painting is likely to be
close to a hierarchy though. Nothing is accidental, everything is related. Yet it
is not a single thing, for that would be boring. Neither is it a heterarchy, for that
would mean chaos. A painting is like a tightly woven cloth. (Figure 30 shows an
example that even became a powerful Gestalt in the public eye.)
314
Koenderink, Gestalts and Pictorial Worlds
The structure of the partial order is likely to relate to the intuitive notion of the
“quality” (in the aesthetic sense) of works of art. It certainly relates to issues of
“composition” (Rudolf Arnheim 1974).
Fig. 30 Grant Wood’s American Gothic (1930, Art Institute of Chicago) is a perhaps deceptively
simple, but actually complicated picture with a rich, and yet tight hierarchical structure. Notice
how the pitchfork is repeated in the window, the men’s trousers, and his face. This painting is
such a strong Gestalt in the public eye that it has been (and still is) parodied numerous times. It is
on a par with the Mona Lisa, the Venus of Milo, and the Raising of the Flag at Iwo Jima.
The nexus of Gestalt relations cannot be mapped on a purely spatial mereology.
The reason is that many Gestalts are not localizable, think of “gist,” or “style”
for instance. Moreover, the iconogenesis constructs space and time from scratch
at every systole. It is not as if there were an inner canvas on which iconogenesis
paints its pictures. There are no such inner pictures, and neither is there a fixed
canvas that acts as a coordinating frame. The optical structures sampled by the
early visual system significantly underdetermine the irradiance distribution at
the retina. This is evident from the familiar “crowding” phenomenon (Herman
Bouma 1970). Tarachopic ambliopia (“scrambled vision,” Robert Hess 1982) is
to some degree present in the generic visual system. As a consequence, infinitely
many “images” would satisfactorily “explain” the structure available in the
nervous system. Iconogenesis has a large degree of freedom.
Especially in the peripheral visual field, the relative locations of remarkable
irradiance gradients (often termed “edges,” and so forth) are only approximately
sampled. The iconogenetic process easily deals with this though, as is evident
315
GESTALT THEORY, Vol. 33, No.3/4
from the fact that one easily “reads” pictures that have been locally scrambled to
a surprising degree. In many cases it is actually hard to notice such disturbances.
Visual artists often play on this remarkable ability. Examples abound in the
backgrounds of baroque and rococo paintings. A striking modern example is
Umberto Boccioni’s La strada entra nella casa (the street enters the house) of
1911. (See figure 31.)
Fig. 31 Umberto Boccioni La strada entra nella casa. For one thing it illustrates the apparent
disarray in the peripheral visual field. This is a tightly knit picture with a complicated Gestalt
hierarchy.
The mereological structure is a mental entity, not a mere description of the “colors
assembled in a certain order.” For any given picture it may well be different for
different presentations, thus different for different observers, or different for the
same person at different times. This cannot be otherwise, since the visual Gestalts
are constructions of the iconogenetic process, similar to biology’s “releasers”
(Tinbergen 1951). Of course persons with similar cultural background and
education are likely to agree to some extent in their iconogenetic processes.
It should also be understood that the notion of a mereology can only be
approximate. Virtually all of the Gestalts making up the mereological structure are
ephemeral. The whole structure fluctuates as the systolic process of iconogenesis
pumps out its presentations.
316
Koenderink, Gestalts and Pictorial Worlds
Conclusion
A pictorial world is a Gestalt composed of a nexus of nested and juxtaposed
Gestalts. That is to say it is quality and meaning through and through. There is
nothing in immediate awareness that is not quality or meaning. Indeed, there is
no way mere structure could exist in awareness, for that would effectively equate
awareness with the representation of physical structure.
The physical world is composed of collections of “pointer readings,” glued
together by abstract, formal theories (Eddington 1929). For the physicist the
understanding is in the glue, a mental picture of the world. The “truth” of the
picture derives from the experience that it allows one to predict potential pointer
readings from actual ones. The pictures are not unique, nor established for all
eternity.
Notice that the qualities and meanings of awareness live on a variety of disparate
levels. These various levels in visual awareness may be related to a basic bottleneck
in the total amount of complexity that can be grasped in a single presentation.
This is apparently very constricted. One has to think of a capacity of less than
half a dozen items (George Miller 1956).
The visual experience of the specious present is temporally rather more expanded
(at least an order of magnitude) than that of a single presentation. The presentations
in a specious present are not necessarily experienced in a fixed sequential order,
but they occur as distinct entities.
Moreover, in extended viewing of a picture you build up a presentational awareness
that is even more expanded (perhaps by another order of magnitude). This is
important, one’s experience of a painting is not limited by a single presentation,
or specious present. The specious presents in a presentational awareness tend to
be in a well defined sequential order.
Of course this means that the experience of a painting should not be understood
as some kind of “image” in a literal sense (A strange notion anyway!). If anything
image-like, it would be like a (perhaps large) number of images. This reminds one
a bit of the way academic artists plan their works by preparing a large number of
preliminary studies or thumbnail sketches. Moreover, the “seeing of a painting”
is like a temporally extended itinerary, rather than a single happening. It is an
exploration that may lead to a variety of discoveries, depending equally on the
explorer, and on the work.
In the case of paintings any single presentation only yields part of an articulated
(the process never really ends) nexus of Gestalts. This is somewhat different from
the conventional examples presented in the literature of experimental psychology.
Such examples have such a limited complexity that they can often be exhausted
in just a few, or even a single, presentation, certainly in a specious present. Most
317
GESTALT THEORY, Vol. 33, No.3/4
paintings are far too highly structured for that. The casual visitor of a museum
may grant a painting only a cursory glance, but the connoisseur knows it takes
time, often an appreciable amount, to get to know a work. Moreover, the best
paintings appear to be inexhaustible, and reward repeated viewing.
In a mere glance one necessarily “zooms in” on just a partial aspect. This is
empirically evident from the recent literature on “change blindness” (Daniel
Simons & Ronald Rensink 2005). What exactly pops up in visual awareness
depends largely on where the eyes happened to be looking, and where the
eyes were looking was largely determined involuntarily by the iconogenetic
process itself, although some fixations are of a voluntary nature. The pattern
of involuntary fixations is different for people with different backgrounds, say
artists, connoisseurs or laymen.
Many artists are convinced that they are able to control the pattern of involuntary
fixations through their composition (Hamm 1988). It is essentially unknown
whether they are right in this, or whether they are simply deceiving themselves.
(Even if they do, it might still be a useful device in planning a painting, of
course). One thing that is certain is that the structure of the nexus of Gestalts in
the presentational awareness of a moderately complex painting is fully beyond
the understanding of current psychological science.
The optical constraint that co-determines a single presentation is largely
determined by the structure of the visual system. In the human, the eyes are
frontally positioned, and move in synchrony. The visual field of a single eye is
about a half-space, but only the innermost center is a high resolution area, the
periphery has a structure that grows progressively coarser as the angular distance
from the direction of view increases. This “structure” is not just resolution,
but also a partial “loss of local sign” (Koenderink 1984; Benjamin Balas et al.
2009), that is a progressive loss of localization of significant local structure.
Although highly important, and conceptually interesting, these structures are
still insufficiently understood in vision science.
Apart from such sampling structures that are attached to the visual direction,
there is the effect of bilaterality, which is related to the attitudes of head and
body. The vertical is clearly different from the horizontal, and the bilateral halves
of the visual field (as determined by the vertical) appear to function somehow
independently of each other. The whole system is dominated by the bilateral
frontal structure of the optical system, the direction of gravity, and the fact that
humans tend to move over the ground-plane in the frontal direction. The effect
of this structure is visible in most pictures, and imposes an overall “Gestalt,” that
is typically taken for granted. Just consider a bunch of pictures for some generic
topic (“landscape,” and so forth) for a Google search, and notice how they are
similarly oriented. Next consider a number of Paul Cezanne’s still life paintings,
318
Koenderink, Gestalts and Pictorial Worlds
and notice the differences (and evident “break”) between the left and right halffields.
In conclusion, the “pictorial worlds” should be understood in terms of qualities and
meanings, Gestalts playing an important structuring role. They are not just “parts”
(even less atomic parts!) but act as both context, and component on a variety of
levels. The formal description of such systems has not even been started yet.
That progress has perhaps been less than impressive, is for a large part due to
the fact that the immediate content of visual awareness, the presentations, is
pre-cognitive. Many of the essential meanings and qualities cannot really be put
into words. A comic book structure may well be more apt than the conventional
scientific format of this journal. Part of the structure of pictures can be described
as one treats narrative (Maurice Denis’ “some anecdote”), but the truly visual
structure cannot. In that sense it is similar to (though very different in substance
from) music, or poetry.
Such problems also extend to the empirical domain. The conventional methods
of cognitive science fall far short of accessing the essential structure of immediate
visual awareness. Recent attempts to get at the content of presentations (Li FeiFei et al. 2007) are interesting, but (necessarily) problematic because of stimuli
and methods. The stimuli are random snap-shots from the Internet, but the
notion of a “random photograph” is void, there is no such a thing. This is obvious
from the fact that averages of many shap-shots of standard touristic targets are
easily recognizable. (The artist Jason Salavon has spectacular examples, just try
Google.) The methods involve verbalization, which is obviously very problematic,
and short exposures. But a short exposure elicits presentations that are quite
different from the momentary presentations in a prolonged viewing. Perhaps the
“change blindness” experiments (Ronald Rensink 2010) are more revealing. Here
one forces a sharp cut between specious presents, and notices what is different in
the experiences immediately before and after the cut. Another paradigm applies
gradual changes, too slow to notice in any one specious moment. Here one may
monitor the limit of unnoticeability.
My present attempt at a synthetic view of the structure of pictorial worlds is
evidently immature, and mainly intended to fire up novel research.
Acknowledgements
The empirical work on pictorial space was done with Johan Wagemans (Laboratorium
voor Experimentele Psychologie, K.U. Leuven) and Andrea van Doorn (Industrial
Design, Delft University of Technology). This work was supported by the Methusalem
program by the Flemish Government (METH/08/02), awarded to Johan Wagemans.
319
GESTALT THEORY, Vol. 33, No.3/4
Summary
Arrangements of colors on planar surfaces often evoke an awareness of “pictorial worlds.”
Obviously imaginary, pictorial worlds have spatial structure, whereas pictorial objects
have shapes, material properties, histories and futures, and stand in causal relations to
each other. Pictorial qualities and meanings simply “happen” in immediate awareness.
They are pre-cognitive, perhaps proto-rational. Pictorial worlds may be very highly
structured. For many paintings one readily distinguishes juxtapositions of hierarchies
of nested Gestalts in which items derive meaning from a higher level, whereas items
at a lower level appear as undifferentiated structural elements or textural quality. At
any given level items co-determine each other. They may apparently causally affect
each other. Structures on the picture surface (oval marks, line segments, dots, …)
acquire many different meanings and qualities. Hierarchies are themselves Gestalts,
simultaneously defining, and being defined by their components. The “grammar,” if
there can be said to be one, is the juxtaposed, and nested order of mutually defining
entities. Qualities and meanings exist on many levels, e.g., in a painting, from “style,” to
the overall “gist,” all the way down to the quality of the brush-strokes.
Keywords: Gestalt hierarchy, visual arts, pictorial grammar, visual literacy.
Zusammenfassung
Farbarrangements auf ebenen Flächen rufen oft eine Vorstellung von „bildhaften Welten“
hervor. Solche offensichtlich imaginären Bildwelten weisen ein räumliche Struktur
auf, während die bildlichen Objekte Formen, Materialeigenschaften, Geschichte und
Zukunft zu haben scheinen, und offensichtlich in kausalem Zusammenhang zueinander
stehen. Bildhafte Qualitäten und Bedeutungen drängen sich zwingend der Vorstellung
auf. Sie sind als prä-kognitiv, und vielleicht auch proto-rational anzusehen. Bildwelten
sind oft stark strukturiert. Bei manchen Bildern meint man ein buntes Zueinander von
Hierarchien ineinander verschachtelter Gestalten an zu treffen, in denen Gegenstände
auf einer höheren Ebene Sinn ergeben, während Elemente auf einer niedrigeren Ebene
als Strukturelemente oder strukturelle Qualitäten auftreten. Auf jeder gegebenen Stufe
bedingen sich diese Elemente gegenseitig, sie können sich anscheinend auch kausal
beeinflussen. Strukturen auf der Bildoberfläche wie Flecke, Striche, oder Punkte
erlangen viele verschiedene Bedeutungen und Qualitäten. Hierarchien sind selbst
Gestalten, die in sich sowohl Ursache (Sinn durch Übergestalten), als auch Wirkung
(Inhalt durch Untergestalten) vereinen. Solche Über- und Unter-Gestalten sind virtuell
vorhanden, da sich in der momentanen Vorstellung jeweils nur eine einzelne Ebene
ausbildet. Eine „Bildgrammatik“, wenn man überhaupt von einer solchen sprechen
kann, ist eine formelle Beschreibung dieser Nebeneinander und Ineinander gestellten,
sich gegenseitig definierenden Gebilde. Qualitäten und Bedeutungen existieren abhängig von der momentanen Vorstellung - auf verschiedenen Ebenen, zum Beispiel
- in einem Gemälde - vom „Stil“ bis zur Gesamtwirkung, und herunter bis zur Qualität
der einzelnen Pinselstriche.
Schlüsselwörter: Gestalthierarchie, bildende Kunst, bildhafte Grammatik, visuelle
Lesefähigkeit.
320
Koenderink, Gestalts and Pictorial Worlds
References
Alberti, L. B. (1435): Della Pittura. Both Latin and Italian texts available at http://www.liberliber.it/
biblioteca/a/alberti/de_pictura/html/index.htm; English translation: http://www.noteaccess.com/Texts/
Alberti/ (20.03.2011).
Arnheim, R. (1966): Toward a psychology of art. Berkeley, CA: University of California Press.
Arnheim, R. (1974): Art and visual perception: A psychology of the creative eye. Berkeley, CA: University of
California Press.
Arnheim, R. (1986): New essays on the psychology of art. Berkeley, CA: University of California Press.
Arnheim, R. (1943): Gestalt and art. Journal of Aesthetics 2, 71-75.
Avgerinou, M. & Ericson, J. (1997): A review of the concept of visual literacy. British Journal of Educational
Technology 28 (4), 280-291.
Balas, B., Nakano, L. & Rosenholtz, R. (2009): A summary-statistic representation in peripheral vision
explains visual crowding. Journal of Vision 9 (13), 11-18.
Berkeley, G. (1709): An Essay Towards a New Theory of Vision. Dublin: Printed by Aaron Rhames, at the Back
of Dick’s Coffee-House, for Jeremy Pepyat, Bookseller in Skinner-Row. Text available online.
Boccioni, U. (1914): Pittura e scultura futuriste (dinamismo plastico. Quoted in a Wikipedia entry http://
en.wikipedia.org/wiki/Umberto_Boccioni (20.03.2011).
Bouma, H. (1970): Interaction effects in parafoveal letter recognition. Nature 241, 177-178.
Brentano, F. (1874): Psychologie vom empirischen Standpunkt. Leipzig: Verlag von Duncker & Humblot.
Brown, J. W. (1996): Time, Will, and Mental Process. New York: Plenum Press.
Clark, K. (1949): Landscape into Art. London: John Murray.
Cook, F. H. (1977): Hua-Yen Buddhism: The Jewel Net of Indra. University Park, PA: Penn State Press.
Crane, W. (1900): Line and form. London: G. Bell & Sons, Ltd.
DaCosta Kaufmann, T. (2010): Arcimboldo: Visual Jokes, Natural History, and Still-Life Painting. Chicago:
University of Chicago Press.
Da Vinci, L. (1835): Treatise on painting. Available on the web http://www.archive.org/details/
davincionpainting00leon (20.03.2011).
Dali, S. (1948): 50 secrets of magic crafsmanship. Chevalier, H. M. (transl.). New York: The Dial Press.
Degas, E. (w.r.): Quoted on http://www.andrewgrahamdixon.com/archive/readArticle/873 (20.03.011).
Denis, M. (1890): Définition du Néo-traditionalisme. Revue Art et Critique, august 30, 1890.
Dennett, D (1991): Consciousness explained. Allen Lane, ed., London: The Penguin Press.
Eddington, A. (1929): The nature of the physical world. Many editions, text available online.
Edwards, B. (1999): The new drawing on the right side of the brain. New York: Jeremy P. Tarcher/Putnam.
Fei-Fei, L., Iyer, A., Koch, C. & Perona, P. (2007): What do we perceive in a glance of areal-world scene?
Journal of Vision 7 (1), 1-29.
Gauß, C.F. (1827): Disquisitiones generales circa superficies curvas. Commentationes Societatis Regiae
Scientiarum Gottingesis Recetiores 6, 99-146.
Gibson, J.J. (1950): The Perception of the Visual World. Boston: Houghton Mifflin.
Gombrich, E.H. (1961): Art and Illusion: A Study in the Psychology of Pictorial Representation. Princeton, NJ:
Princeton University Press.
Greene, M.R. & Oliva, A. (2009): Recognition of Natural Scenes from Global Properties: Seeing the Forest
Without Representing the Trees. Cognitive Psychology 58 (2), 137-179.
Hamm, J. (1988): Drawing scenery: Landscapes and seascapes. New York: Perigee Trade.
Hamm, J. (1963): Drawing the head and figure. New York: Perigee Trade.
Harrison, M. A. (1978): Introduction to Formal Language Theory. Reading, Mass.: Addison-Wesley Publishing
Company.
Helmholtz, H. von (1977): Epistemological writings. IV.~The facts in perception. Appendix 1. On the localization
of the sensations of internal organs, in Cohen, R.S. & Elkana, Y. (eds.), Boston Studies in the Philosophy of
Science XXXVIII. Dordrecht Holland/Boston USA: D. Reidel Publishing Company.
Hendrickx, M. & Wagemans, J. (1999): A critique of Leyton‘s theory of perception and cognition. Journal of
Mathematical Psychology 43, 314-345.
Hess, R.F. (1982): Developmental sensory impairment: amblyopia or tarachopia. Human Neurobiology 1,
17–29.
Hildebrand, A. von (1901): Das Problem der Form in der bildenden Kunst. (3rd ed.) Available online.
Hoffman, D.D. (1998): Visual Intelligence, How We Create What We See. New York: W. W. Norton &
Company, Inc.
321
GESTALT THEORY, Vol. 33, No.3/4
Hoffman, D.D. (2008): Sensory experiences as cryptic symbols of a multi-modal user interface, in Bauer, M.,
Liptay, F. & Marschall, S. (eds.): Kunst und Kognition. Munich: Wilhelm Fink, 261-279.
Hoffman, D.D. (2009): The interface theory of perception: Natural selection drives true perception to swift
extinction, in Dickinson, S., Tarr, M., Leonardis, A. & Schiele, B. (eds.): Object categorization: Computer
and human vision perspectives. Cambridge UK: Cambridge University Press), 148-165.
Jacobs, T.S. (1988): Light for the artist. New York: Watson-Guptill.
James, W. (1893): The principles of psychology. New York: H. Holt and Company.
Jastrow, J. (1899) The mind‘s eye. Popular Science Monthly 54, 299-312.
Koenderink, J.J. (1984): The concept of local sign, in van Doorn, A. J., van de Grind, W. A. & Koenderink,
J. J. (eds.): Limits in Perception. Utrecht: VNU~Science~Press.
Koenderink, J.J. (1990): Solid Shape. Cambridge MA: MIT Press.
Koenderink, J.J. & van Doorn, A.J. (1992): Surface shape and curvature scales. Image and Vision
Computing 10, 557-565.
Koenderink J.J. & van Doorn A.J. (2000): Blur and disorder. Journal of Visual Communication and Image
Representation 11, 237-244.
Koenderink, J.J. (2011): Vision and Information, in Albertazzi, L., van Tonder, G. J. & Vishwanath, D.
(eds.): Perception beyond Inference. The Information Content of Visual Processes. Cambridge, MA: MIT
Press, Bradford book, 27-57.
Koenderink, J.J. & van Doorn, A. J. (2003): Shape and shading, in Chalupa, L. M. & Werner, J. S. (eds.):
The visual neurosciences, 1090-1105. Cambridge, MA: MIT Press.
Koenderink, J.J., van Doorn, A. J. & Wagemans, J. (2011). Depth. i-Perception special issue, in press.
Leibniz, G.W. (1991, original 1714): La Monadologie. Edition établie par E. Boutroux. Paris: LGF.
Leśniewski, S. (1992): Collected Works. Surma, S.J., Srzednicki, J.T., Barnett, D.I., & Rickey, V.F., editors
and translators. Norwell, MA: Kluwer.
Leyton, M. (1992): Symmetry, Causality, Mind. Cambridge MA, MIT Press.
Lorenz, K. (1973): Die Rückseite des Spiegels. München: Piper Verlag.
Luderin, P. (1997): L’Art pompier: immagini, significati, presenze dell‘altro Ottocento francese (1860-1890).
Pocket library of studies in art, Firenze: Olschki.
Meinong, A. (1899): Über Gegenstände höherer Ordnung und deren Verhältniss zur inneren Wahrnehmung.
Zeitschrift für Psychologie und Physiologie der Sinnesorgane 21, 187–272.
Mellem, J. (2009): Sketching people. Life drawing basics. Cincinnati, Ohio: North Light Books.
Palmer, S.E. (1999): Vision science: Photons to phenomenology. Cambridge, MA: Bradford Books/MIT Press.
Merleau-Ponty, M. (1993): Eye and mind, in Edie, J.M. (ed.), Dallery, C. (transl.): The primacy of perception.
Evanston: Northwestern University Press.
Metzger, W. (1975): Gesetze des Sehens. Frankfurt a. M.: Verlag Waldemar Kramer.
Miller, G.A. (1956): The magical number seven, plus or minus two: Some limits on our capacity for
processing information. Psychological Review 63 (2): 343–355.
Miller, F. (1998): Frank Miller’s Sin City, booze, broads, & bullets. Milwaukee, Oregon: Dark Horse Books
Necker, L.A. (1832): Observations on some remarkable optical phænomena seen in Switzerland; and on
an optical phænomenon which occurs on viewing a figure of a crystal or geometrical solid. London and
Edinburgh Philosophical Magazine and Journal of Science 1 (5), 329–337.
Oliva, A. & Torralba, A. (2001): Modeling the Shape of the Scene: a Holistic Representation of the Spatial
Envelope. International Journal in Computer Vision 42, 145-175.
Panofski, E. (1925): Die Perspektive als symbolische Form. Vorträge der Bibliothek Warburg, 1925, 258330, published separately, Die Perspektive als symbolische Form. Leipzig and Berlin: B. G. Teubner, 1927,
English, Perspective as Symbolic Form. New York: Zone Books, 1991.
Pepperell, R. (2006): Seeing without objects: Visual indeterminacy and art. Leonardo 30(5), 394-400.
Piles, R. de (1708): Cours de peinture par principes avec un balance de peintres. Available on the Internet.
Pinker, S. (1994): The language instinct. New York: Morrow.
Pinna, B. (2010): New Gestalt Principles of Perceptual Organization: An Extension from Grouping to Shape
and Meaning. Gestalt Theory – An International Multidisciplinary Journal 32 (1), 11-78.
Pinna, B. & Albertazzi, L. (2011): From grouping to visual meanings: a new theory of perceptual organization.
In Albertazzi, L., van Tonder, G. J. & Vishwanath, D. (eds.): Perception beyond Inference. The Information
Content of Visual Processes. Cambridge, MA: MIT Press, Bradford book, 287-344.
Ramachandran, V.S. (1988): Perceiving shape from shading. Scientific American 259, 58-65.
Rensink, R.A. (2010): Seeing seeing. Psyche 16(1), 68-78.
Riedl, R. (1984): Biology of knowledge: The evolutionary basis of reason. Chichester, U.K.: John Wiley & Sons.
322
Koenderink, Gestalts and Pictorial Worlds
Rimmer, W. (1970, Original 1877): Art anatomy. Mineola, NY: Dover Publications.
Rogers, L.R. (1969): Sculpture, the appreciation of the arts/2. London: Oxford University Press.
Routley, R. (1982): Exploring Meinong‘s Jungle and Beyond. Atascadero, CA: Ridgeview Pub Co.
Sayce, A.H. (1911): Grammar, in Chisholm, Hugh: Encyclopædia Brittannica (Eleventh ed.). Cambridge,
UK: Cambridge University Press.
Schröder, B.S.W. (2003): Ordered Sets: An Introduction. Boston: Birkhäuser.
Schrödinger, E. (1992; original 1944): What is Life? with Mind and Matter and Autobiographical Sketches.
Cambridge, UK: Cambridge University Press, Canto Editio.
Shaftesbury, A., Earl of (2000, original 1714): Characteristics of Men, Manners, Opinions, Times. Cambridge,
UK: Cambridge University Press.
Shannon, C.E. (1948): A mathematical theory of communication. Bell System Technical Journal 27, 379–423
& 623–656.
Simons, D.J. & Rensink, R.A. (2005): Change blindness: Past, present, and future. Trends in Cognitive
Sciences 9(1), 16-20.
Speed, H. (1917): The practice & science of drawing. London: Seeley, Service and Co., Ltd.
Stieglitz, A. (1923): How I came to Photograph Clouds. Amateur Photographer and Photography 56 (1816),
255.
Teuber, M.L. (1973): New aspects of Paul Klee‘s Bauhaus style. In Paul Klee: Paintings and watercolors from
the Bauhaus years, 1921-1931. Des Moines, IA: Des Moines Art Center.
Teuber, M.L. (1976): Blue Night by Paul Klee, in Mary Henle, ed.: Vision and artifact: Essays in honor of
Rudolf Arnheim. New York: Springer, 131-151.
Teuber, M.L. (1974): Sources of ambiguity in the prints of Maurits C. Escher. Scientific American 231, 90104.
Tinbergen, N. (1951): The study of instinct. London, U.K.: Oxford Clarendon Press.
Unrein, S. (2011): Website www.suzanneunrein.com (20.03.2011).
Uexküll, J. von (1909): Umwelt und Innenwelt der Tiere. Berlin: Springer.
van Doorn, A. J., Koenderink, J. J. & Wagemans, J. (2011a). Light fields and shape from shading. Journal of
Vision, 11(3), 21, 1-21.
van Doorn, A. J., Koenderink, J. J. & Wagemans, J. (2011b). Rank order scaling of pictorial depth.
i-Perception special issue. Revised manuscript submitted.
Verstijnen, I. & Wagemans, J. (2004). Ambiguous figures: Living versus nonliving objects. Perception 33,
531-546.
Wagemans, J., van Doorn, A. J. & Koenderink, J. J. (2010). The shading cue in context. i-Perception 1, 159178.
Wagemans, J., van Doorn, A. J. & Koenderink, J. J. (2011). Measuring 3-D point configurations in pictorial
space. i-Perception 2(1), 77-111.
White, M. (1963): Equivalence: The Perennial Trend. PSA Journal 29(7), 17–21
Jan Koenderink, (born 1943) studied physics, mathematics and astronomy at Utrecht University, where he
graduated in1972. From the late 1970’s he held a chair „The Physics of Man“ at Utrecht University till his
retirement in 2008. He presently is Research Fellow at Delft University of Technology, and guest professor at
Leuven University. He is a member of the Dutch Royal Society of Arts and Sciences, and received a honorific
doctorate in medicine from Leuven University. Current interests include the mathematics and psychophysics
of space and form in vision, including applications in art and design.
Address: Prof. Dr. Jan J. Koenderink, Katholieke Universiteit Leuven, Laboratorium voor Experimentel
Psychologie, Tiensestraat 102, bus 3711, 3000 Leuven, België, and Delft University of Technology, Man
Machine Interaction Group, EEMCS, P.O. Box 5031, 2600 GA Delft, The Netherlands
E-Mail: j.j.koenderink@tudelft.nl
323
Gestalttheoretische Inspirationen
Anwendungen der Gestalttheorie
(Handbuch zur Gestalttheorie • Band 2)
Herausgegeben von Hellmuth Metz-Göckel
246 Seiten, € 25,-ISBN 978 3 901811 59 3
Die Anfänge der Gestalttheorie reichen in die 30er Jahre des vorigen Jahrhunderts zurück. Seitdem hat sie zahlreiche Weiterentwicklungen erfahren
und gilt mittlerweile als bedeutende Metatheorie nicht nur der Psychologie, sondern auch vieler anderer Disziplinen.
Die Beiträge des vorliegenden Bandes sollen demonstrieren, welch fruchtbare Anregungen von der Gestalttheorie auf andere Disziplinen ausgehen
können. Der Band vereinigt Anwendungen gestalttheoretischen Denkens
auf die Bereiche Philosophie und Phänomenologie, Psychotherapie, Kunsttheorie und Pädagogik, soziale Tugenden und soziales Denken, Design, Literatur, Sprache, Fotografie sowie Tierpsychologie.
Fax: + 43 1 985 21 19-15 | Mail: verlag@krammerbuch.at