Three-dimensional shape coding in grasping circuits: a
comparison between the Anterior Intraparietal area and ventral
premotor area F5a
Tom Theys1, 2, Pierpaolo Pani1, Johannes van Loon2, Jan Goffin2, Peter Janssen1
Laboratorium voor Neuro- en Psychofysiologie1 and Afdeling Experimentele
Neurochirurgie en Neuroanatomie2, Katholieke Universiteit Leuven
Herestraat 49
B-3000 Leuven
Belgium
Corresponding author:
Peter Janssen MD PhD
Laboratorium voor Neuro- en Psychofysiologie, Leuven Medical School
Herestraat 49, bus 1021
B-3000 Leuven
Belgium
Phone: +32 16 34 57 45
Fax: +32 16 34 59 93
Email: peter.janssen@med.kuleuven.be
Abstract
Depth information is necessary for adjusting the hand to the three-dimensional shape of
an object in order to grasp it. The transformation of visual information into appropriate
distal motor commands is critically dependent on the Anterior IntraParietal area (AIP)
and the ventral premotor cortex (area F5), particularly the F5p sector. Recent studies
have demonstrated that both AIP and the F5a sector of the ventral premotor cortex
contain neurons that respond selectively to disparity-defined three-dimensional (3D)
shape. To investigate the neural coding of 3D shape and the behavioral role of 3D-shape
selective neurons in these two areas, we recorded single-cell activity in AIP and F5a
during passive fixation of curved surfaces and during grasping of real-world
objects.Similar to those in AIP, F5a neurons were either first- or second-order disparity
selective, frequently showed selectivity for discrete approximations of smoothly-curved
surfaces that contained disparity discontinuities, and exhibited mostly monotonic tuning
for the degree of disparity variation. Furthermore, in both areas, 3D-shape selective
neurons were co-localized with neurons that were active during grasping of real-world
objects. Thus area AIP and F5a contain highly similar representations of 3D shape,
which is consistent with the proposed transfer of object information from AIP to the
motor system through the ventral premotor cortex.
Introduction
Binocular disparity based on differences in the horizontal positions of retinal images
provides an important cue for three-dimensional (3D) object recognition and manipulation.
Indeed, binocular disparity information is used for object manipulation while the hand is
preshaping to adapt to the intrinsic 3D properties of an object (Watt & Bradshaw, 2003).
The Anterior Intraparietal area (AIP) and the ventral premotor cortex are considered
critical nodes in the visual control of grasping in non-human primates based on single-cell,
inactivation and functional imaging results (Rizzolatti et al., 1988; Taira, Mine,
Georgopoulos, Murata, & Sakata, 1990; Sakata & Kusunoki, 1992; Gallese, Murata, Kaseda,
Niki, & Sakata, 1994; Jeannerod, Arbib, Rizzolatti, & Sakata, 1995; Sakata, Taira, Kusunoki,
Murata, & Tanaka, 1997; Fogassi et al., 2001; Fluet, Baumann, & Scherberger, 2010;
Nelissen & Vanduffel, 2011). Human imaging studies have reported grasp-related activations
in the human AIP and the ventral premotor cortex (Faillenot, Toni, Decety, Gregoire, &
Jeannerod, 1997; Binkofski et al., 1998; Jacobs, Danielmeier, & Frey, 2010). In line with
early behavioral work on grasping (Jeannerod, 1981), a recent human fMRI study
demonstrated activations in the human AIP and ventral premotor cortex elicited by intrinsic
rather than extrinsic object properties (Cavina-Pratesi et al., 2010).
We previously demonstrated that neurons in area AIP and area F5a, which is adjacent
to F5p, provide a robust representation of 3D shape (Srivastava, Orban, De Maziere, &
Janssen, 2009; Theys, Pani, van Loon, Goffin, & Janssen, 2012). The presence of 3D-shape
selective neurons in F5a does not indicate how these neurons represent disparity-defined 3D
shapes, i.e. which aspects of the stimuli are primarily encoded by the neurons. Previous
studies have shown that the coding of 3D shape in area AIP is fast (i.e. short latency), metric
(i.e. largely monotonic tuning for the degree of the disparity variation in curved surfaces) and
coarse (i.e. selectivity is retained for discrete approximations (Srivastava et al., 2009)). In
contrast, 3D shape coding in the inferior temporal cortex (ITC) is slower (longer latencies),
more categorical, and highly sensitive to disparity discontinuities such as sharp edges and
disparity steps (Janssen, Vogels, & Orban, 2000b). Similar coding differences between dorsal
and ventral stream areas have been suggested by human fMRI studies (Preston, Li, Kourtzi, &
Welchman, 2008). These differences in the neural representation of 3D shape between ITC
and AIP are consistent with lesion studies in humans (Goodale & Milner, 1992), which have
proposed a dichotomy in the primate visual system between the ventral stream supporting
perception/object recognition and the dorsal stream supporting action (Dijkerman, Milner, &
Carey, 1996; Marotta, Behrmann, & Goodale, 1997): the very detailed representation of 3D
shape in ITC is suitable for object recognition and categorization, whereas the coarser
representation of 3D shape in AIP is suitable for grasping.
We have also recently shown that the 3D-shape representation in AIP is more
boundary-based (i.e. weak influence of surface dots and robust selectivity for stimuli with
disparity varying along the boundaries of the shape) compared to that in ITC (Theys,
Srivastava, van Loon, Goffin, & Janssen, 2012). However, it is unknown whether the 3Dshape representation in F5a is more similar to the one in AIP or that in the ITC. Given that
F5a is strongly connected to AIP but not directly with ITC (Matelli, Camarda, Glickstein, &
Rizzolatti, 1986; Gerbella, Belmalih, Borra, Rozzi, & Luppino, 2011) we hypothesized that
the neural coding of 3D shape in F5a should be more similar to that in AIP than in ITC.
Recent studies have also begun to investigate the behavioral role of 3D-shape selective
neurons in ITC and F5a. Consistent with the properties of individual ITC neurons (Janssen et
al., 2000b), electrical microstimulation of ITC clusters during 3D-shape categorization exerts
a profound and predictable influence on both perceptual choices and reaction times (Verhoef,
Vogels, & Janssen, 2012). In the F5a sector of ventral premotor cortex, in contrast, most 3D-
shape selective neurons were also active during visually-guided grasping of real-world objects
(Theys et al., 2012), suggesting a close relationship between 3D-shape selectivity and
visuomotor transformations for grasping. However, no data exist concerning the possible
behavioral role of 3D-shape selective sites in AIP.
First-order disparity coding has been demonstrated in areas V4 (Hinkle & Connor,
2002) and ITC (Janssen, Vogels, & Orban, 1999) in the ventral stream, as well as in areas
MT/V5 (Nguyenkim & DeAngelis, 2003), CIP (Shikata, Tanaka, Nakamura, Taira, & Sakata,
1996; Taira, Tsutsui, Jiang, Yara, & Sakata, 2000; Katsuyama et al., 2010) and AIP
(Srivastava et al., 2009) in the dorsal stream. Second-order disparity selectivity has been
demonstrated in ITC (Janssen et al., 2000b) and AIP (Srivastava et al., 2009). The first goal of
the present study was to investigate the neural representation of 3D shape in F5a to compare
this representation with that of AIP. We tested whether F5a neurons respond to first-order (as
in tilted planar surfaces) or second-order disparity (curved surfaces). We also tested the
coarseness of the 3D-shape coding in F5a by comparing the selectivity for smoothly curved
surfaces to that for discrete and linear approximations of these surfaces containing disparity
discontinuities, and by examining the sensitivity of F5a neurons for small differences in the
degree of the disparity variation. Finally, to complement our previous study of 3D-shape
selectivity and grasping activity in F5a (Theys et al., 2012) we also recorded from 3D-shape
selective sites in AIP during visually-guided grasping, and trained both animals in a
perceptual 3D-shape categorization task, to be able to relate interindividual differences in the
neural representations of 3D shape to behavioral differences in discrimination performance.
To allow a direct comparison between F5a and area AIP, we recorded in both areas (in
counterbalanced order) in the same animals using the same stimulus sets and tasks. We found
that the 3D-shape representation in F5a was highly similar to that in AIP. Conversely, 3Dshape selective sites in AIP were also frequently active during object grasping, as in F5a.
Thus, the parietofrontal grasping circuit contains two almost identical representations of the
depth structure of objects that are both closely related to the visual guidance of the hand
during grasping.
Materials and Methods
Subjects, surgery and recording procedure
Two rhesus monkeys served as subjects for extracellular micro-electrode
recordings. The surgical and recording procedures were identical to previous experiments in
ITC and AIP (Janssen et al., 1999; Janssen, Vogels, & Orban, 2000a; Janssen et al., 2000b;
Srivastava et al., 2009). Under isoflurane anesthesia, a magnetic resonance imaging (MRI)compatible head fixation post and recording wells were implanted. Implantation of the
recording chambers was centered on the inferior limb of the arcuate sulcus (area F5) and the
lateral bank of the anterior intraparietal sulcus (AIP) stereotactically guided using
preoperative MRI. All surgical techniques and veterinary care were performed in accordance
with the NIH Guide for Care and Use of Laboratory Animals and approved by the local
ethical committee of the KU Leuven. The animals were trained to perform a passive fixation
task keeping the gaze of both eyes inside a 1-degree fixation window. After a 400-ms fixation
period, the stimulus was presented at the fixation point for 600 ms, and if fixation had been
maintained, a drop of juice was given as a reward. Stimuli were presented dichoptically using
ferroelectric liquid crystal shutters (Displaytech) operating at a frequency of 60 Hz each and
synchronized with the vertical retrace of the display monitor (VRG) operating at 120 Hz. As
previously reported (Srivastava et al., 2009), no crosstalk was measured between the images
presented to the two eyes. Horizontal and vertical eye movements were recorded using an
infrared-based camera system sampling at 500 Hz (EyeLink II; SR Research).
Stimuli and tests
The basic stimulus set used in the search test was identical to the one used in previous
experiments (Janssen et al., 2000b; Srivastava et al., 2009) and consisted of disparity-defined
3D shapes generated by combining a two-dimensional contour and a depth profile. We used
thirty-two 3D shapes portraying curved surfaces, in which four disparity profiles were
imposed over eight 2D shapes filled with a 50% density random-dot pattern (stimulus size:
5.5 deg). For all subsequent tests, stimuli with opposite curvatures were created by
interchanging the monocular images between eyes (concave surfaces become convex and vice
versa). All neurons were therefore tested with pairs of 3D shapes (concave-convex) composed
of the same monocular images, but with right and left images interchanged.
Recordings in area AIP and F5a were performed according to previous experiments
(Srivastava et al., 2009; Verhoef, Vogels, & Janssen, 2010; Theys et al., 2012). In the search
test - which was identical to that used in our previous studies - we presented the basic
stimulus set consisting of 32 stimuli (eight 2D contours combined with 4 depth profiles,
degree of the disparity variation, i.e. the disparity difference between the center and the edge
of the shape: 0.65 deg) at the fixation point during passive fixation. On the basis of responses
obtained in this test, the optimal 2D contour was selected for all subsequent tests. As in our
previous studies, we first verified that the selectivity for 3D shape could not be accounted for
by a selectivity for the monocular images per se (disparity test, see (Theys et al., 2012)): if the
response difference between the two members of a pair of 3D shapes (composed of the same
monocular images) was significant (t-test p < 0.05) and at least three times greater than the
difference in the sum of the monocular responses (i.e. the difference in the summed responses
to the monocular presentations of each of the two members of a pair of 3D shapes), the
neuron was judged to be disparity selective. When stereo selectivity was present, we assessed
that selectivity for higher-order disparity in the position-in-depth test (for further details, see
(Janssen et al., 2000b)) in which concave and convex 3D stimuli were presented at five
positions in depth, ranging from - 0.50° (near) to + 0.50° (far).
If a neuron in area F5a was judged to be higher-order disparity-selective, the cell was
further tested in the disparity order test to assess its selectivity for first- and second-order
disparity using various approximations of the original 3D shapes. This stimulus set has been
described in Janssen et al. (Janssen et al., 2000b) and Srivastava et al.(Srivastava et al., 2009).
Briefly, the first-order stimulus approximations consisted of planar surfaces inclined in depth
(linear variation in disparity over the vertical axis of the shape), to approximate either the top
or the bottom part of the original three-dimensional shape (Fig. 1B). The linear
approximations of the concave and convex 3D shape pairs were derived from the original 3D
shapes as least-squares approximations of disparity profiles in the original surfaces, and
consisted of two tilted planes (linear disparity variations) with a sharp disparity discontinuity
in the center of the stimulus. For the inclined 3D shape pair (see example neuron in Figure 2)
the linear approximation was a single inclined surface and therefore identical to the first-order
approximation. We tested linear approximations only for the cosine and Gaussian depth
profiles, as in Srivastava et al. (Srivastava et al., 2009). Three different discrete
approximations were constructed by dividing the stimulus into three parts with a central
region of varying size. These three parts were presented at three positions in depth (disparity
amplitude, 0.65°).
In the disparity sensitivity test, the optimal 2D contour was presented with two depth
profiles (e.g. concave and convex) and six degrees of disparity variation (i.e. the disparity
difference between the center and the edge of the shape) ranging from 1.3 to 0.03 deg.
Because our goal was to compare the representation of 3D shape in F5a with that in AIP, all
stimuli were presented in the center of the display at the fixation point and in the fixation
plane. M2 was also used in the AIP study of Srivastava et al. (Srivastava et al., 2009). To
allow direct comparison between AIP and F5a we also recorded from higher order disparityselective neurons in the sensitivity test in area AIP of monkey M1.
To investigate the selectivity of F5a neurons for 3D shapes in which the disparity
variation was confined to the boundary or the surface of the stimulus, we presented the same
stimuli as in previous studies (Janssen, Vogels, Liu, & Orban, 2001; Theys et al., 2012).
Different Gaussian depth profiles were created by varying the disparity along the surface of
the stimulus, along the boundary, or along both surface and boundary (boundary-surface test).
In the surface stimuli the depth profile consisted of a 2D-gaussian or a 13° x 13° square with
the maximum disparity in the center of the shape smoothly approaching zero towards the
boundaries.
A subset of 3D-shape selective neurons was tested in a visually-guided grasping task
(described in Theys et al (Theys et al., 2012)). In this task, the monkey was seated in the dark
and had to place its right hand in a resting position for 500 ms, after which an LED was
illuminated at the bottom of an object. Fixation upon the LED (keeping the gaze inside a 2.5degree fixation window throughout the trial until the object was lifted) for 500 ms was
followed by the illumination of a 3D object. One of six different objects (a small cylinder, a
small cube, a large cylinder, a large sphere, a large cube and a cylinder with groove) was
pseudorandomly presented on a custom-built, vertically rotating carousel at a viewing
distance of 28 cm. The dimensions (width, length and height) of the small and large objects
were 15 and 35 mm, respectively. After a variable period of 500 to 1000 ms of object fixation,
an auditory GO-cue instructed the monkey to reach, grasp, and lift the object. After a
correctly completed trial, a juice reward was given. The resting position and the lifting phase
were monitored by fiber-optic cables. Eye movements were recorded using an infrared-
sensitive camera system (EyeLink II; SR Research). Eye position signals, neural activity, and
photocell pulses were digitized and processed with a digital signal processor (DSP) at 20 kHz
(C6000 series; Texas Instruments, Dallas, TX, USA).
Recording procedure
For extracellular recordings, tungsten micro-electrodes (FHC, MicroProbes) were
inserted through a guide tube placed in a standard grid (Crist Instruments). Magnetic
resonance imaging (resolution 0.6 mm isotropic) using glass capillaries filled with a 1%
copper sulfate solution inserted into key grid positions confirmed correct positioning of the
electrode in the depth of the posterior bank of the inferior limb of the arcuate sulcus (area
F5a) and in the lateral bank of the anterior intraparietal sulcus (area AIP). For area F5a, we
reconstructed the electrode penetrations based on the anatomical MRI using BrainVISA
(http://brainvisa.info). Because of the inclination of the recording cylinder, we first traversed
area 45B on the anterior bank of the inferior arcuate sulcus, then passed through the sulcus
and finally entered the posterior bank of the inferior arcuate sulcus near the fundus. The
patterns of active and silent zones during the recordings were highly consistent with the
reconstructed electrode paths. The center of our recording area was located within the fMRI
activation elicited by curved surfaces reported by Joly et al. (Joly, Vanduffel, & Orban, 2009).
The recording positions in F5a were identical to the ones in our previous study (Theys et al.,
2012); microstimulation in these recording positions did not elicit any overt behavioral
response. The AIP recordings were performed in posterior AIP, in a region where Sakata et al.
reported hand-manipulation neurons (Sakata, Taira, Murata, & Mine, 1995).
Data analysis
Matlab (Mathworks) was used for data analysis. Net neural responses were calculated
by subtracting the mean activity in the 400 ms interval preceding stimulus onset from the
mean activity between 50 and 450 ms after stimulus onset. In the position-in-depth test,
neurons were considered responsive to the spatial variation of disparity (i.e. higher-order
disparity selective) if the response to the non-preferred shape did not significantly (Student’s
t-test) exceed any response to the preferred shape at any position in depth (Janssen et al.,
2000b).
In the disparity-order test, neurons were classified as first order if the selectivity for
the first-order stimuli was not significantly smaller than the selectivity for the original
smoothly curved surfaces, as evidenced by a nonsignificant interaction between threedimensional structure (concave-convex) and stimulus type (original vs first-order stimulus,
ANOVA p > 0.05). Second-order neurons were significantly more selective for the smoothly
curved surfaces compared to the first-order approximations (ANOVA p < 0.05). Discrete
neurons showed a significant (t-test, p < 0.05) selectivity for at least one of the discrete
approximations.
Depending on the tuning for 3D shape as determined in the sensitivity test, neurons
were classified as monotonic, broadband or tuned as in Janssen et al., (2000b). Monotonic
neurons showed the strongest response to the largest disparity variation or a response
statistically indistinguishable (t-test) from the response to the second-largest disparity
variation, and a decline in response for smaller disparities. Broadband neurons responded
equally well to all disparity variations (no significant effect of degree of disparity variation in
a one-way ANOVA of the responses to the six disparity variations of the preferred 3D shape).
If an optimal disparity variation was found, with a significant decrease in response on either
side of this optimal magnitude, the neuron was classified as tuned.
In the grasping task, neural activity (MUA) was aligned to the onset of the
illumination above the object, to the time of the go-signal and to the time of the object lift.
Results
All F5a neurons reported here (N = 131) were higher-order disparity selective, i.e.
showed significant selectivity in the disparity test (consisting of presentations of concave and
convex surfaces and monocular presentations) that could not be accounted for by the
monocular responses and preserved this selectivity across positions-in-depth. Binocular eye
position traces recorded during the position-in-depth test showed only marginal deviations
(averaging 0.1 degree of vergence) between the nearest and the farthest position in depth,
much smaller than the range of disparities in the position-in-depth test (1 deg). Although
visual responses could be recorded over an extended range in and around the arcuate sulcus,
higher-order disparity selective neurons were concentrated in area F5a, in line with previous
fMRI results (Joly et al., 2009).
Disparity order test
To determine whether F5a neurons encode first- or second-order disparities, we tested
71 F5a (M1: N = 35; M2: N = 36) neurons in the disparity-order test, in which the original 3D
shapes were presented together with various approximations of these stimuli. The example
neuron in Fig. 2A was highly selective for the inclined depth profile (first column), but was
equally selective for a simple first-order approximation of the original 3D shape pair (second
column, ANOVA with factors 3D profile and stimulus type, interaction ns). This example
neuron was therefore encoding primarily first-order variations in disparity, as present in tilted
planes. Furthermore, the discrete approximations (three rightmost columns) also elicited
significant selectivity (t-tests on the net responses, p < 0.01 for all three approximations),
indicating that the coding of 3D shape was relatively coarse.
Across our population of F5a neurons, 55% (39/71; M1: N = 14/35; M2: N = 25/36)
showed selectivity for first-order stimuli equal to or greater than that for second-order stimuli,
and were therefore considered first-order neurons, in line with previous studies (Janssen et al.,
2000b; Srivastava et al., 2009). The average responses in the disparity-order tests of these
first-order neurons are illustrated in Fig. 2B. Clearly both the first-order and the discrete
approximations evoked strongly selective responses in this population of first-order neurons
(t-test on the net responses to the preferred and nonpreferred depth profiles, p < 0.01 for every
approximation).
However, we also observed strong second-order selectivity in F5a, as illustrated by the
example neuron in Fig. 3A. This neuron was strongly selective for convex versus concave
depth profiles and for their linear approximations, but not for the first-order approximations
(rightmost column; ANOVA with factors 3D shape and stimulus type, interaction p < 0.05).
Two out of three discrete approximations also elicited significant selectivity (t-test, p < 0.01),
which again indicates that the representation of 3D shape in F5a is relatively coarse.
Neurons showing significantly less selectivity for the first-order stimuli compared to
the second-order stimuli (ANOVA, interaction between 3D profile and stimulus type p <
0.05) were deemed to be second-order disparity selective. In the average response of our
population of second-order disparity selective F5a neurons (N = 32/71, 45%) (Fig. 3B), no
selectivity was present for the first-order stimuli. As in the example neuron in Fig. 3A, two out
of three discrete approximations yielded significant selectivity across the population (t-test p
< 0.05), and 19 neurons (59%) showed selectivity for at least one of the discrete
approximations. Interestingly, the average response to the preferred linear approximation
differed significantly from the response to the preferred smoothly-curved 3D shape (t-test, p <
0.01; M1: p < 0.001; M2: p < 0.05), and 12 neurons discriminated reliably between the
preferred smoothly-curved 3D shape and its linear approximation. Therefore the neural
coding of 3D shape in F5a is, on the one hand, relatively coarse (in view of the frequent
selectivity for discrete approximations), but at the same time sensitive enough to signal the
small differences in 3D profile between the smoothly curved surface and its linear
approximation, which constitutes a least-squares approximation of the smoothly curved
surface. Note that we also encountered 17 neurons that were selective for the full-period sine
depth profile, consisting of combined convex and concave profiles (see Fig. 2 in Theys et
al.,(Theys et al., 2012)) and that therefore represents a third-order disparity stimulus. However
we did not perform the disparity-order test for these neurons.
To illustrate the responses and the neural selectivity for first- and second-order stimuli
we plotted the average net responses and response differences for the original preferred 3D
shape against the responses and response differences for the first-order stimuli (Fig. 4A), for
the first-order and second-order neurons independently. Although second-order neurons
frequently fired strongly to first-order stimuli (left panel in Fig. 4A), the response differences
were much smaller for first-order stimuli than for second-order stimuli (right panel in Fig.
4A). Similarly, discrete approximations can evoke strong responses in second-order F5a
neurons (left panel in Fig. 4B), but the degree of selectivity was weaker for these discrete
approximations than for the smoothly curved 3D surfaces (right panel in Fig. 4B). Finally and
in contrast to area AIP, a substantial proportion of F5a neurons (38%) discriminated reliably
between a smoothly curved surface and its linear, least-square approximation (see data points
below the diagonal in Fig. 4C).
Overall, F5a neurons can encode zero-, first-, second- and possibly third-order
disparities, similar to those in AIP and ITC. The neural coding of 3D shape in F5a shares
many features with that in area AIP (e.g. selectivity for discrete approximations), but a subset
of the neurons in F5a appear to be more sensitive to subtle differences in the depth profile
compared to AIP neurons.
Sensitivity to the degree of disparity variation
To investigate the precision of the 3D-shape representation in area F5a, we measured
the sensitivity of F5a neurons to differences in the degree of the disparity variation within the
stimulus (an amplitude of 1.3 deg corresponded to a highly-curved surface, whereas an
amplitude of 0.03 deg was almost flat). In the disparity sensitivity test, 131 neurons (M1: 61;
M2: 70) were tested with disparity variations in the stimulus ranging from 1.3 to 0.03 deg.
Fig. 5 shows the responses of three example neurons (positive numbers on the X-axis indicate
the preferred depth profile). The most frequent response pattern in both monkeys (72%; M1:
79%, M2: 66%; z-test, ns; Table 1) was a monotonic profile, with the maximal response to the
largest disparity variation and a monotonic decline in the response for smaller disparity
variations (green line in Fig. 5). A small proportion of the neurons (8%; M1: 13%, M2: 4%; ztest, ns; Table 1) were significantly tuned to a particular disparity variation: the maximal
response was observed for one of the smaller disparity variations, and this response differed
significantly (t-test p < 0.05) from the response to the largest disparity variation (blue trace in
Fig. 5). Finally 20% of the neurons (M1: 8%; M2: 30%; Table 1) were broadband (red trace in
Fig. 5) since no significant difference was observed between responses in the preferred range
of disparity variations (ANOVA, ns). The proportion of broadband neurons was significantly
greater in M2 than in M1 (z-test, p < 0.05). However, the strongest decline in the response
was more frequently (38% of the broadband neurons) seen within the nonpreferred range of
disparity variations (as illustrated by the example neuron in Fig. 5) than at the change in the
sign of the curvature (11%), in contrast to what has been reported in ITC (Janssen et al.,
2000b).
Although most F5a neurons showed a monotonic response pattern in the sensitivity
test, a fraction of these neurons displayed significant selectivity for small differences in the
depth profiles of the stimuli: more than 20 % of F5a neurons (28 % in M1 and 23 % in M2)
were significantly selective for the smallest differences in the depth profile (+0.03 vs -0.03
deg, t-test p < 0.05) and 18 out of 131 neurons tested (15%, M1: 18%, M2: 11%) showed the
largest response difference at the change in the sign of the disparity curvature (i.e. between
the two smallest disparity variations, -0.03 vs +0.03 deg). Our population of 131 F5a neurons
reliably discriminated between the two smallest disparity variations (paired t-test of the
average responses to the +0.03 and -0.03 deg disparity variations, p < 0.0001 for both
monkeys combined; M1: p < 0.001; M2: p = 0.12). Thus, F5a neurons can signal very small
differences in the depth profiles of curved surfaces.
The average normalized responses in the disparity sensitivity test are illustrated in Fig.
6A for both monkeys independently. In both monkeys, the F5a population showed a largely
monotonic response pattern, but in M1 a more pronounced drop in the response was present
between the two smallest disparity variations (t-test comparing the difference in the
normalized response differences between the -0.03 and +0.03 deg disparity variations
between the two monkeys, p < 0.001). To directly compare the representation of 3D shape in
F5a with that in AIP, we also recorded the responses of 71 higher-order AIP neurons in the
same animals (N = 38 in M1, N = 33 in M2). The average normalized responses of AIP
neurons are illustrated in Fig. 6B. Overall the average response in AIP was highly similar to
that of F5a. Furthermore, the proportions of monotonic, broadband and tuned neurons were
highly similar in AIP and F5a in both animals: also, monotonic neurons predominated in AIP
(M1: 74%; M2: 85%), whereas tuned (M1: 10%; M2: 9%) and broadband (M1: 16%; M2:
6%) neurons represented much smaller fractions of the neurons. In both monkeys the
normalized difference between responses to the -0.03 and +0.03 deg disparity variations did
not significantly differ in the two areas (t-test, M1: p = 0.46; M2: p = 0.82). Therefore the
results of the sensitivity test indicate that the neural representation of 3D shape in F5a was
highly similar to that in AIP.
To determine whether the difference between the neural representations of 3D shape in
our two monkeys was related to interindividual differences in the quality of stereoscopic
vision, we trained both animals in a 3D-shape discrimination task (Verhoef et al., 2010;
Verhoef, Vogels, & Janssen, 2011; Verhoef et al., 2012) In this task, either a convex or
concave 3D surface (disparity varied along both the vertical and the horizontal axis, no
disparity on the boundaries (Theys et al., 2012) was presented at the fixation point, and the
animal was required to make an eye movement to the left when the stimulus was concave and
to the right when the stimulus was convex. The disparity coherence was always 100%. After 6
training sessions, M1 reached a performance level of 82% correct, whereas M2 still
performed at chance level (50% correct) after 12 training sessions. The chance performance
of M2 was not due to an inability to learn the task rule, since this animal had learned a simple
shape discrimination task (saccade to the left for a square and to the right for a triangle) in 5
sessions (92 % correct). Given the presence of large numbers of disparity-selective neurons in
F5a and AIP (Srivastava et al., 2009) in M2, it is unlikely that this animal was stereoblind, but
behavioral testing indicated that the quality of its stereoscopic perception was in all likelihood
weaker than that of M1. Therefore the differences we observed between our two animals in
the disparity sensitivity test were likely related to differences in stereoscopic perception.
Selectivity for surfaces and boundaries in depth
We previously demonstrated that 3D-shape selective AIP neurons encode both
disparity variations along the boundary and along the surface of the shape, and that for the
great majority of AIP neurons, boundaries in depth (lacking 3D surface information) are
sufficient for evoking selectivity (Theys et al., 2012). We tested 18 higher-order F5a neurons
with the same stimuli as in our previous study (Theys et al., 2012): concave and convex
curved surfaces with a disparity variation on both the surface and the boundary of the shape
(vertical 3D shape), on the boundary of the shape but not on the surface (silhouettes and
outline stimuli), and on the surface but not on the boundary (restricted and large 3D surfaces).
In a manner very similar to those of AIP, most (78%) F5a neurons were selective for at least
one of the curved boundaries, whereas a smaller proportion (55%) was selective for the 3D
surface stimuli (data not shown). Although the low number of neurons precludes a detailed
comparison between AIP and F5a in terms of 3D boundary selectivity, the proportions of
neurons in F5a were highly comparable to those previously reported for AIP (67% boundary
neurons, 53% surface neurons).
3D-shape selectivity and grasping activity
As a final comparison of 3D-shape selective sites in F5a and AIP, we recorded multiunit activity (MUA) in 3D-shape selective AIP sites during delayed visually-guided object
grasping after assessing higher-order disparity selectivity in the position-in-depth test, as in
our previous study of F5a (Theys et al., 2012). We found that the great majority of the 3Dshape selective sites (13/16, 81%) also responded during the fixation and grasping of realworld objects. Because our AIP recordings consisted of MUA, we cannot infer that the same
AIP neurons were both 3D-shape selective and active during grasping. Furthermore we did
not test whether 3D-shape selective AIP sites remained active during grasping in the dark
(visuomotor activity), as in F5a. However, at the very least these data demonstrate that 3Dshape selectivity was co-localized with grasping activity in AIP, as it is in F5a(Theys et al.,
2012).
In Figure 7, the average population response during visually-guided grasping is plotted
as a function of time for area AIP (top panel). For comparison we also plotted the average
grasping-related activity of 98 higher-order disparity selective MUA sites in F5a (bottom
panel). The visual response in the first 100 ms after light onset appeared stronger and faster in
AIP in comparison to F5a (Fig. 7 left panels), but because the latency of the population
response strongly depends on the number of recording sites, we cannot directly compare the
two areas in this respect. At the moment of object lift ([-100, 200 ms] around object lift) the
average activity in AIP declined and the average F5a activity became stronger than in AIP. A
mixed-design ANOVA with brain area (AIP – F5a) and epoch ([-100, 200 ms] around object
lift and [-500, -200 ms] before object lift) as independent factors revealed a significant
interaction (F(1,108) = 8,18, p = 0.005). Bonferroni post-hoc analysis showed a significant
difference (p < 0.002) between neural activity in the [-100, 200 ms] epoch around the lift and
that in the epoch [-500, -200 ms] before the lift (respectively 39.9 ±8.3 spikes/sec vs 9.46 ±7.6
spikes/sec) for AIP, while no difference was observed between these epochs for F5a (24.3
(±3.3) vs 18.75 (±3.0)). Although the low number of AIP sites warrants a degree of caution in
interpreting these results, they may suggest that 3D-shape selective AIP sites are most active
during the visual analysis of the object, whereas 3D-shape selective F5a sites are more
strongly active during the execution of the grasping movement. Future studies will have to
determine to what extent 3D-shape selective AIP neurons remain active during grasping in the
dark, i.e. exhibit visuomotor or motor-dominant activity.
Discussion
We investigated the coding of 3D shape selective neurons in area F5a and compared
their properties with AIP neurons. We found that F5a neurons could be either first-order or
second-order disparity selective. Furthermore, 3D shape coding in area F5a was fast, robust,
largely metric and coarse, similar to area AIP. In both areas, 3D-shape selective neurons were
embedded in clusters of neurons that also fired during grasping. The coding of 3D shape
information in AIP and F5a is most likely important for translating 3D object properties into
the appropriate motor commands for grasping.
This study is the first detailed comparison between the object representation in AIP
and F5a. It is remarkable that neurons in the ventral premotor cortex provide exceedingly
detailed visual information about the 3D-structure of objects (e.g. the selectivity for very
small disparity variations and the differences between smoothly curved surfaces and their
linear approximation). Our data also demonstrate that at least some F5a neurons encode not
only relative disparity (i.e. the disparity difference between the center and the edge of the
shape) but also first- and second-order disparities (curvature). Hence the motor system has
access to a robust visual 3D description of objects, even though this information may not
necessarily determine the grip type. Murata et al. (Murata, Gallese, Luppino, Kaseda, &
Sakata, 2000) and Raos et al. (Raos, Umilta, Murata, Fogassi, & Gallese, 2006) compared the
object representations in AIP and F5 using multidimensional scaling and cluster analysis, and
concluded that AIP furnishes a visual object description whereas F5 represents objects in
motor terms (i.e. determined by the grip type used to grasp the object). However, the study of
Raos et al. (Raos et al., 2006) focused on the F5p sector in ventral premotor cortex and most
likely did not include F5a neurons.
It is noteworthy that the 3D-shape stimuli we used did not resemble the objects that
the monkeys had to grasp; nevertheless the same ensembles of neurons were active in F5a and
AIP during 3D-shape presentation and during object grasping. At least part of this overlap in
neural preference for these seemingly disparate stimulus classes may arise from the relatively
broad tuning for objects in AIP and F5a, since most neurons in these areas respond to many
objects (Pani et al., unpublished observations). Moreover, in addition to the 3D profile, AIP
and F5a neurons also encode the 2D contour of objects, which may be based on relatively
simple shape features that are shared between many objects. Also in the ITC, it is frequently
difficult to identify a single object or shape feature that activates the neuron, and many
neurons appear to respond to seemingly unrelated shape features, even in the 3D domain
(Janssen et al., 2001).
We have previously investigated the neural coding of 3D shape in the ITC and in AIP
using the same stimuli (Janssen et al., 2000b; Srivastava et al., 2009). The neural
representation of 3D shape information was very similar in areas F5a and AIP. The premotor
area F5a contained many neurons for which a discrete approximation was sufficient to evoke
selectivity, and was similar to area AIP in this regard (Srivastava et al., 2009). The sensitivity
to the degree of the disparity variation with an overall monotonic coding and the selectivity
for minute disparity differences in a small fraction of the neurons was also very similar to
AIP. However, we observed a tendency towards a more elaborate and detailed representation
in area F5a since in both monkeys, neurons were signaling subtle distinctions in depth profiles
such as the difference between a linear approximation (a wedge-shape stimulus) and a
smoothly-curved 3D shape, which was not observed in AIP (Srivastava et al., 2009). This
refinement suggests that additional processing occurs between the output of AIP and the
output of F5a before visual information connects to the motor system. Finally, in both F5a and
AIP, 3D-shape selective neurons were embedded in clusters of neurons that were also active
during visually-guided grasping. The latter observation highlights the probable behavioral role
of 3D-shape selective neurons in the parietal and premotor cortex, i.e. to provide a visual 3D
description of objects for the purpose of programming the grasp. Consistent with this idea,
reversible inactivation of 3D-shape selective sites in AIP causes a marked deficit in grasping
but not in the perceptual discrimination of 3D-structure (Verhoef, Vogels and Janssen,
unpublished observations).
However, we also observed similarities between ITC, AIP and F5a in 3D-shape
coding. We observed zero-order (position in depth), first-order (tilt/slant) and second-order
disparity selectivity (curvature) in all three areas. In contrast, earlier visual areas such as V4
contain zero- and first-order neurons (Hinkle & Connor, 2002) but no second-order neurons
(Hegde & Van Essen, 2005). Hence the properties of neurons in lower-tier areas appear to be
reiterated at the highest levels in dorsal, ventral and premotor areas. Furthermore, in a manner
similar to neurons in area AIP (Theys et al.) and area IT (Janssen et al., 2001) F5a cells
showed selectivity for three-dimensional surfaces and boundaries.
The anatomical connectivity between areas AIP and F5a (Gerbella et al., 2011), the
response latency difference between AIP and F5a (approximately 10 ms, (Theys et al., 2012)),
and the strong resemblance in functional properties (fast and coarse coding, grasping-related
activity) suggest a hierarchical parietofrontal 3D processing network for the control of
grasping, distinct from the ventral stream 3D-shape representation in IT. Whether the
selectivity in area F5a depends exclusively on input from area AIP needs further confirmation
through combined recording-inactivation experiments. The current physiological and
anatomical evidence suggests that AIP input could be processed in area F5a and translated
into a more motoric code for the F5p neurons which project to M1 and the spinal cord
(Gerbella et al., 2011). Consistent with this hypothesis, virtual lesions of the human AIP using
theta-burst Transcranial Magnetic Stimulation (TMS) disrupt the normal PMv-M1
interactions during grasp preparation (Davare, Rothwell, & Lemon, 2010), suggesting that
these PMv-M1 interactions depend on the object information provided by AIP to the premotor
cortex.
Human fMRI studies indicate that part of the PMv is also activated more strongly by
curved surfaces than by flat surfaces at different positions in depth (Georgieva, Peeters,
Kolster, Todd, & Orban, 2009), which could be homologous to the F5a region of the monkey.
On the other hand, the homology between the human AIP (hAIP) (Culham & Kanwisher,
2001; Culham et al., 2003; Frey, Vinton, Norlund, & Grafton, 2005; Begliomini, Wall, Smith,
& Castiello, 2007; Cavina-Pratesi, Goodale, & Culham, 2007) and the monkey AIP may be
more questionable. We recorded in the posterior part of AIP (Srivastava et al., 2009), in which
strong visual (3D-shape selective) and grasping responses can be measured but no
somatosensory responses are found (Pani, Theys and Janssen, unpublished observations). A
range of fMRI studies in humans and monkeys (reviewed in (Orban, 2011) suggest that
posterior AIP in the monkey may correspond more to the DIPSA region in the human, which
is located in the IPS posterior to the hAIP and is also activated by 3D shape defined by
disparity (Durand, Peeters, Norman, Todd, & Orban, 2009). The hAIP, in contrast, is more
activated during grasping than during reaching (Culham et al., 2003), responds even to
somatosensory stimulation (Bodegard, Geyer, Grefkes, Zilles, & Roland, 2001), but is not (or
only weakly) activated by disparity-defined 3D shape (Durand et al., 2009), similar to the
more anterior part of AIP. [Note that DIPSA may also be activated during grasping, but not
more than during reaching because of the strong visual responses in this region.] Furthermore
Culham et al. (Culham et al., 2003) reported that the hAIP is not activated by images of
objects, whereas we observed strong and selective responses to images of objects in the
macaque AIP (Romero, Van Dromme, & Janssen, 2012). These apparently conflicting results
between human fMRI and single-cell studies may have been caused by the homology between
the macaque posterior AIP and a more posterior region in the human IPS (DIPSA) resulting
from the expansion of the human IPS areas compared to the monkey. A similar reasoning may
apply for the homology between the human Lateral Occipital Complex (LOC (Malach et al.,
1995; Grill-Spector, Kourtzi, & Kanwisher, 2001) and monkey ITC.
The question remains as to why two separate but very similar visual representations of
3D shape should exist in both AIP and F5a for the control of grasping. One explanation could
involve differences in neural properties that we did not test for, such as different receptive
field sizes. Another possibility is that F5a neurons could respond to larger object parts or
more complex shape features than AIP neurons, comparable to the V4 – TEO – TE hierarchy
in the ventral stream (Kourtzi & Connor, 2011). The observation that F5a neurons, but not
AIP neurons, signal the difference between a smoothly curved 3D shape and its linear
approximation suggests that the 3D-shape representation in F5a may be more refined than in
AIP and that additional processing is required before connecting to the motor system.
Extensive receptive field mapping and a systematic variation of shape features to determine
their critical features in each of these two areas could answer such questions. Multi-unit
recordings in F5a revealed that 3D-shape selective visual-dominant neurons are co-localized
with visuomotor neurons that were active during grasping in the light and in the dark (Theys
et al., 2012). Furthermore F5p/c neurons showed sluggish and nonselective responses to our
3D-shape stimuli (Theys et al., 2012), and previous studies have demonstrated that objects are
encoded in motor terms rather than in visual terms in F5p (Raos et al., 2006). Therefore we
hypothesize that the main reason for an intermediate area (F5a) between AIP and F5p may be
that the connection between visual information and motor commands occurs in these clusters
of visual-dominant and visuomotor neurons in F5a which then project to F5p. In this
interpretation the grasping-related activity we observed in 3D-shape selective clusters of AIP
would be entirely different in nature: motor-related grasping activity in AIP may arise as a
corollary discharge from visuomotor and motor-dominant F5p neurons projecting back to AIP
for online visual control (Rizzolatti & Luppino, 2001), a hypothesis that deserves
experimental testing.
We also observed interindividual differences between the two animals in the disparity
sensitivity test: the average F5a and AIP population response in M1 showed a larger decline at
the point where the disparity curvature changes sign (between the 0.03 deg preferred
amplitude and 0.03 deg non-preferred amplitude) compared to monkey M2. The observed
difference in neural sensitivities was most likely related to a difference in the quality of
stereoscopic vision in the two animals, since M1 learned to discriminate disparity-defined
curved surfaces in a limited number of training sessions whereas M2 did not. However, M2
was not stereoblind, since we recorded large numbers of disparity-selective neurons in this
animal’s AIP and F5a. Furthermore, we measured a normal stereo visually-evoked potential
(VEP) over V1 with disparity stimuli in this animal (Janssen, Vogels, & Orban, 1998;
Srivastava et al., 2009). In humans also, a wide range of stereoscopic capacities and
stereoanomalies can be observed (Howard & Rogers, 2002). Individual differences in neural
sensitivity that we observed suggest that differences between cortical areas have to be
interpreted cautiously if the data are not acquired in the same animals. However, our two
monkeys were quite comparable in the proportions of zero-, first- and second-order neurons,
as well as in the proportion of neurons showing selectivity for at least one of the discrete
stimuli. Furthermore, in both animals the average AIP population response in the sensitivity
test was largely monotonic.
Previous monkey fMRI studies found stronger activations for curved surfaces
compared to flat surfaces at different positions-in-depth which were highly localized in F5a
(Joly et al., 2009), but very extensive and comprising a large part of AIP and the anterior
region of LIP (Durand et al., 2007) in the lateral bank of the IPS. However in our single-cell
recording experiments, the extent of the recording area that contained 3D-shape selective
neurons was highly similar in AIP and F5a, matching closely the F5a activation described in
Joly et al. (Joly et al., 2009). Future studies will have to investigate the neural basis of the
difference between F5a and AIP in the hemodynamic response to curved surfaces.
Acknowledgments
We thank Piet Kayenbergh, Gerrit Meulemans, Stijn Verstraeten, Marc Depaep, Wouter
Depuydt and Inez Puttemans for assistance, and Steve Raiguel for comments on the
manuscript.
Grants
This work was supported by Geconcerteerde Onderzoeksacties (GOA 2005/18, 2010/19),
Fonds voor Wetenschappelijk Onderzoek Vlaanderen grants (G.0495.05, G.0713.09),
Excellentiefinanciering (EF05/014), Programmafinanciering (PFV/10/008) and ERC-StG260607.
Disclosure
The authors report no conflicts of interest.
Reference List
Begliomini, C., Wall, M. B., Smith, A. T., & Castiello, U. (2007). Differential cortical activity
for precision and whole-hand visually guided grasping in humans. Eur.J.Neurosci., 25,
1245-1252.
Binkofski, F., Dohle, C., Posse, S., Stephan, K. M., Hefter, H., Seitz, R. J. et al. (1998).
Human anterior intraparietal area subserves prehension: a combined lesion and
functional MRI activation study. Neurology, 50, 1253-1259.
Bodegard, A., Geyer, S., Grefkes, C., Zilles, K., & Roland, P. E. (2001). Hierarchical
processing of tactile shape in the human brain. Neuron, 31, 317-328.
Cavina-Pratesi, C., Goodale, M. A., & Culham, J. C. (2007). FMRI reveals a dissociation
between grasping and perceiving the size of real 3D objects. PLoS.One., 2, e424.
Cavina-Pratesi, C., Monaco, S., Fattori, P., Galletti, C., McAdam, T. D., Quinlan, D. J. et al.
(2010). Functional magnetic resonance imaging reveals the neural substrates of arm
transport and grip formation in reach-to-grasp actions in humans. J.Neurosci, 30,
10306-10323.
Culham, J. C., Danckert, S. L., DeSouza, J. F., Gati, J. S., Menon, R. S., & Goodale, M. A.
(2003). Visually guided grasping produces fMRI activation in dorsal but not ventral
stream brain areas. Exp.Brain Res., 153, 180-189.
Culham, J. C., & Kanwisher, N. G. (2001). Neuroimaging of cognitive functions in human
parietal cortex. Curr.Opin.Neurobiol., 11, 157-163.
Davare, M., Rothwell, J. C., & Lemon, R. N. (2010). Causal connectivity between the human
anterior intraparietal area and premotor cortex during grasp. Curr.Biol., 20, 176-181.
Dijkerman, H. C., Milner, A. D., & Carey, D. P. (1996). The perception and prehension of
objects oriented in the depth plane. I. Effects of visual form agnosia. Exp.Brain Res.,
112, 442-451.
Durand, J. B., Nelissen, K., Joly, O., Wardak, C., Todd, J. T., Norman, J. F. et al. (2007).
Anterior regions of monkey parietal cortex process visual 3D shape. Neuron, 55, 493505.
Durand, J. B., Peeters, R., Norman, J. F., Todd, J. T., & Orban, G. A. (2009). Parietal regions
processing visual 3D shape extracted from disparity. Neuroimage., 46, 1114-1126.
Faillenot, I., Toni, I., Decety, J., Gregoire, M. C., & Jeannerod, M. (1997). Visual pathways
for object-oriented action and object recognition: functional anatomy with PET.
Cereb.Cortex, 7, 77-85.
Fluet, M. C., Baumann, M. A., & Scherberger, H. (2010). Context-specific grasp movement
representation in macaque ventral premotor cortex. J.Neurosci, 30, 15175-15184.
Fogassi, L., Gallese, V., Buccino, G., Craighero, L., Fadiga, L., & Rizzolatti, G. (2001).
Cortical mechanism for the visual guidance of hand grasping movements in the
monkey: A reversible inactivation study. Brain, 124, 571-586.
Frey, S. H., Vinton, D., Norlund, R., & Grafton, S. T. (2005). Cortical topography of human
anterior intraparietal cortex active during visually guided grasping. Brain Res.Cogn
Brain Res., 23, 397-405.
Gallese, V., Murata, A., Kaseda, M., Niki, N., & Sakata, H. (1994). Deficit of hand
preshaping after muscimol injection in monkey parietal cortex. Neuroreport, 5, 15251529.
Georgieva, S., Peeters, R., Kolster, H., Todd, J. T., & Orban, G. A. (2009). The processing of
three-dimensional shape from disparity in the human brain. J.Neurosci, 29, 727-742.
Gerbella, M., Belmalih, A., Borra, E., Rozzi, S., & Luppino, G. (2011). Cortical connections
of the anterior (F5a) subdivision of the macaque ventral premotor area F5. Brain
Struct.Funct., 216, 43-65.
Goodale, M. A., & Milner, A. D. (1992). Separate visual pathways for perception and action.
Trends Neurosci., 15, 20-25.
Grill-Spector, K., Kourtzi, Z., & Kanwisher, N. (2001). The lateral occipital complex and its
role in object recognition. Vision Res., 41, 1409-1422.
Hegde, J., & Van Essen, D. C. (2005). Role of primate visual area V4 in the processing of 3-D
shape characteristics defined by disparity. J.Neurophysiol., 94, 2856-2866.
Hinkle, D. A., & Connor, C. E. (2002). Three-dimensional orientation tuning in macaque area
V4. Nat.Neurosci, 5, 665-670.
Howard, I. P., & Rogers, B. J. (2002). Seeing in depth. Toronto: I. PORTEUS.
Jacobs, S., Danielmeier, C., & Frey, S. H. (2010). Human anterior intraparietal and ventral
premotor cortices support representations of grasping with the hand or a novel tool.
J.Cogn Neurosci., 22, 2594-2608.
Janssen, P., Vogels, R., Liu, Y., & Orban, G. A. (2001). Macaque inferior temporal neurons
are selective for three-dimensional boundaries and surfaces. J.Neurosci., 21, 94199429.
Janssen, P., Vogels, R., & Orban, G. A. (1998). Assessment of stereopsis in rhesus monkeys
using visual evoked potentials. Doc.Ophthalmol., 95, 247-255.
Janssen, P., Vogels, R., & Orban, G. A. (1999). Macaque inferior temporal neurons are
selective for disparity-defined three-dimensional shapes. Proc.Natl.Acad.Sci.U.S.A,
96, 8217-8222.
Janssen, P., Vogels, R., & Orban, G. A. (2000a). Selectivity for 3D shape that reveals distinct
areas within macaque inferior temporal cortex. Science, 288, 2054-2056.
Janssen, P., Vogels, R., & Orban, G. A. (2000b). Three-dimensional shape coding in inferior
temporal cortex. Neuron, 27, 385-397.
Jeannerod, M. (1981). Intersegmental coordination during reaching at natural visual objects.
In Attention and Performance IX. Long, J. & Baddeley (Eds.), A.Erlbaum, Hillsdale.
153-168.
Jeannerod, M., Arbib, M. A., Rizzolatti, G., & Sakata, H. (1995). Grasping objects: the
cortical mechanisms of visuomotor transformation. Trends Neurosci., 18, 314-320.
Joly, O., Vanduffel, W., & Orban, G. A. (2009). The monkey ventral premotor cortex
processes 3D shape from disparity. Neuroimage., 47, 262-272.
Katsuyama, N., Yamashita, A., Sawada, K., Naganuma, T., Sakata, H., & Taira, M. (2010).
Functional and histological properties of caudal intraparietal area of macaque monkey.
Neuroscience.
Kourtzi, Z., & Connor, C. E. (2011). Neural representations for object perception: structure,
category, and adaptive coding. Annu.Rev Neurosci, 34, 45-67.
Malach, R., Reppas, J. B., Benson, R. R., Kwong, K. K., Jiang, H., Kennedy, W. A. et al.
(1995). Object-related activity revealed by functional magnetic resonance imaging in
human occipital cortex. Proc.Natl.Acad.Sci.U.S.A, 92, 8135-8139.
Marotta, J. J., Behrmann, M., & Goodale, M. A. (1997). The removal of binocular cues
disrupts the calibration of grasping in patients with visual form agnosia. Exp.Brain
Res., 116, 113-121.
Matelli, M., Camarda, R., Glickstein, M., & Rizzolatti, G. (1986). Afferent and efferent
projections of the inferior area 6 in the macaque monkey. J.Comp Neurol., 251, 281298.
Murata, A., Gallese, V., Luppino, G., Kaseda, M., & Sakata, H. (2000). Selectivity for the
shape, size, and orientation of objects for grasping in neurons of monkey parietal area
AIP. J.Neurophysiol., 83, 2580-2601.
Nelissen, K., & Vanduffel, W. (2011). Grasping-related functional magnetic resonance
imaging brain responses in the macaque monkey. J.Neurosci, 31, 8220-8229.
Nguyenkim, J. D., & DeAngelis, G. C. (2003). Disparity-based coding of three-dimensional
surface orientation by macaque middle temporal neurons. J.Neurosci, 23, 7117-7128.
Orban, G. A. (2011). The extraction of 3D shape in the visual system of human and
nonhuman primates. Annu.Rev.Neurosci., 34, 361-388.
Preston, T. J., Li, S., Kourtzi, Z., & Welchman, A. E. (2008). Multivoxel pattern selectivity
for perceptually relevant binocular disparities in the human brain. J.Neurosci, 28,
11315-11327.
Raos, V., Umilta, M. A., Murata, A., Fogassi, L., & Gallese, V. (2006). Functional properties
of grasping-related neurons in the ventral premotor area F5 of the macaque monkey.
J.Neurophysiol., 95, 709-729.
Rizzolatti, G., Camarda, R., Fogassi, L., Gentilucci, M., Luppino, G., & Matelli, M. (1988).
Functional organization of inferior area 6 in the macaque monkey. II. Area F5 and the
control of distal movements. Exp.Brain Res., 71, 491-507.
Rizzolatti, G., & Luppino, G. (2001). The cortical motor system. Neuron, 31, 889-901.
Romero, M. C., Van Dromme, I., & Janssen, P. (2012). Responses to two-dimensional shapes
in the macaque anterior intraparietal area. Eur.J.Neurosci., 36, 2324-2334.
Sakata, H., & Kusunoki, M. (1992). Organization of space perception: neural representation
of three-dimensional space in the posterior parietal cortex. Curr.Opin.Neurobiol., 2,
170-174.
Sakata, H., Taira, M., Kusunoki, M., Murata, A., & Tanaka, Y. (1997). The TINS Lecture.
The parietal association cortex in depth perception and visual control of hand action.
Trends Neurosci., 20, 350-357.
Sakata, H., Taira, M., Murata, A., & Mine, S. (1995). Neural mechanisms of visual guidance
of hand action in the parietal cortex of the monkey. Cereb.Cortex, 5, 429-438.
Shikata, E., Tanaka, Y., Nakamura, H., Taira, M., & Sakata, H. (1996). Selectivity of the
parietal visual neurones in 3D orientation of surface of stereoscopic stimuli.
Neuroreport, 7, 2389-2394.
Srivastava, S., Orban, G. A., De Maziere, P. A., & Janssen, P. (2009). A distinct
representation of three-dimensional shape in macaque anterior intraparietal area: fast,
metric, and coarse. J.Neurosci., 29, 10613-10626.
Taira, M., Mine, S., Georgopoulos, A. P., Murata, A., & Sakata, H. (1990). Parietal cortex
neurons of the monkey related to the visual guidance of hand movement. Exp.Brain
Res., 83, 29-36.
Taira, M., Tsutsui, K. I., Jiang, M., Yara, K., & Sakata, H. (2000). Parietal neurons represent
surface orientation from the gradient of binocular disparity. J.Neurophysiol., 83, 31403146.
Theys, T., Pani, P., van Loon, J., Goffin, J., & Janssen, P. (2012). Selectivity for threedimensional shape and grasping-related activity in the macaque ventral premotor
cortex. J.Neurosci., 32, 12038-12050.
Theys, T., Srivastava, S., van Loon, J., Goffin, J., & Janssen, P. (2012). Selectivity for threedimensional contours and surfaces in the anterior intraparietal area. J.Neurophysiol.,
107, 995-1008.
Verhoef, B. E., Vogels, R., & Janssen, P. (2010). Contribution of inferior temporal and
posterior parietal activity to three-dimensional shape perception. Curr.Biol., 20, 909913.
Verhoef, B. E., Vogels, R., & Janssen, P. (2011). Synchronization between the end-stages of
the dorsal and the ventral visual stream. J.Neurophysiol..
Verhoef, B. E., Vogels, R., & Janssen, P. (2012). Inferotemporal cortex subserves threedimensional structure categorization. Neuron, 73, 171-182.
Watt, S. J., & Bradshaw, M. F. (2003). The visual control of reaching and grasping: binocular
disparity and motion parallax. J.Exp.Psychol.Hum.Percept.Perform., 29, 404-415.
Figure legends
Figure 1. Recording positions and stimuli.
A. Recording areas. Coronal images illustrating the most representative recording sites in area
F5a and area AIP for monkey M1. The inset shows the estimated recording positions in F5a
and AIP (red areas on the lateral view of the monkey brain).
B. Monocular images presented to the left and right eyes and schematic illustrations of the
perceived 3D structure are shown for one convex 3D shape stimulus, the linear
approximation, one of the discrete approximations and the first-order stimulus approximation.
Note that the only depth cue arises from binocular disparity and that the shading in the figures
is added for illustration only. Exchanging the monocular images between the eyes yielded the
opposite depth profiles.
Figure 2. First-order disparity selectivity in area F5a.
A. Example neuron. PSTH of a first-order F5a neuron showing the responses to the preferred
(top row) and nonpreferred (bottom row) original 3D shape (first column), its linear
approximation (second column, which was also a first-order approximation), and the three
discrete approximations (rightmost columns). The neuron showed robust selectivity for the
inclined three-dimensional shape and for the first-order stimulus. The three discrete
approximations yielded similarly strong selectivity. The vertical calibration bar indicates 100
spikes/s.
B. Population response. Mean (± SEM) net responses to the preferred (open bars) and
nonpreferred (shaded bars) original 3D shape (first column), the linear approximation (second
column) and three discrete approximations (right), for all first-order neurons (N = 39) are
shown. Asterisks indicate significant selectivity in the population (p < 0.01).
Figure 3. Second-order disparity selectivity in area F5a.
A. Example neuron. PSTH of a second-order F5a neuron for the preferred (top row) and
nonpreferred (bottom row) original 3D shape (first column), its linear approximation (second
column), followed by the three discrete approximations, and the first-order approximation
(rightmost column). The vertical calibration bar indicates 100 spikes/s.
B. Population response. Mean net responses to the preferred (open bars) and nonpreferred
(shaded bars) original three-dimensional shape (left), the linear approximation (wedge
stimulus), the three discrete approximations, and the first-order stimuli (right), for all
second-order F5a neurons (N = 32). Asterisks indicate significant selectivity in the population
(p < 0.01).
Figure 4. Disparity order test in F5a: scatterplots
A. The net response (left panel) and the net response difference (right panel) to the best firstorder stimulus is plotted as a function of the response and the response difference to the
original preferred 3D shape for first-order (blue) and second-order (red) neurons.
B. The net response (left) and the net response difference (right) to the best discrete
approximation is plotted against the net response and the net response difference to the
original smoothly curved 3D shape for all second-order neurons (N = 32).
C. The net response (left) and the net response difference (right) to the linear approximation is
plotted against the net response and the net response difference to the original smoothlycurved 3D shape for all second-order neurons (N = 32).
Figure 5. Sensitivity test in F5a: example neurons.
The mean net response (spikes/sec) is plotted as a function of the 12 different degrees of
disparity variation (-1.3  +1.3 deg, positive values were used for the preferred disparity
variation) for three example F5a cells illustrating the different types of neurons: monotonic
(green), tuned (blue) and broadband (red).
Figure 6. Sensitivity test: population analysis.
A. The mean normalized response in M1 (left) and M2 (right) plotted as a function of the 12
different degrees of disparity variation (-1.3  +1.3 deg) for F5a neurons tested in the
sensitivity test (N = 61 for M1 and N = 70 for M2).
B. The mean normalized population response for M1 (left) and M2 (right) plotted as a
function of the 12 different degrees of disparity variation (-1.3  +1.3 deg) for AIP neurons
tested in the sensitivity test (N = 38 for M1 and N = 33 for M2). Positive values indicate the
preferred disparity variation.
Figure 7. Visually-guided grasping: population response.
The average response of a subset of 3D-shape selective neurons in area AIP (N =16) is plotted
as a function of time (during object fixation and reach-to-grasp) after stimulus onset (left),
time of the go-signal (middle) and time of object lift (right panel). For comparison, the
average response of a subset of 3D-shape selective neurons in area F5a (N = 98) is plotted in
the same task.
Tables
Table 1. Sensitivity test: relative proportions (%) of different types of neurons (monotonic,
broadband and tuned neurons) in area F5a and area AIP.
F5a
AIP
M1
M2
M1
M2
Monotonic
79
66
74
85
Broadband
8
30
16
6
Tuned
13
4
10
9
M1: Monkey 1, M2: Monkey 2