ELSEVIER
Ncurocomputing 38-40 (2001) 1595-1602
w~~~w.eIsevics.co~n/locatcl~~e~~co~~~
Perception of change in depth in the hummingbird
hawkmoth Murzdctca sextn (Sphingidne, Lepidopter-a)
M.Wickleina9*, T.J. ~ejnowski"."
"Co/~r/)rr/n/io~rrtl
:Vc,rrrohiolog~~
l,rrhor~~/o~:v.
Hottvrd H~cglresMrclictrl /tr.s/i/~r/c,.
S~rlliI I I . S / ~ / ~ I / L , .
10010 N Tot.rq. Pirtc..~Rd. Lri Jolln CA '12037. US.4
t'~c~l)(~t~ttrtcttt
o/'Biolo,y~,,U r t i r ~ r . . ~of
itj~
Cu/i'~%r~ri(i,
&it7 Diego, Lo . / o l / ~C'.4
~ , '120M, ULS.d
Abstract
Visual perception of depth change can be mediated monocularly by looming the apparent
size increase of a n approaching object. In hlcrrid~rcrrse.~tawe recorded intraccllularly from cells
that detect both approach and rctrcat of an object. The cells compute looming in two
length of
fundamentally difl'ercnt ways: class I neurons measur-c the change of perir~~eter/edgc
the object; class 2 neurons respond to expansion/contraction llowfields. We created a network
model incorpor-ating anatomical and physiological propcrtics of class 1 neurons to understand
the underlying computational principles for looming detection. .:( 1001 Elscvicr Science B.V.
All rights reserved.
Ke>>\vor-(is: Looming:
Monocular depth perception; Visual interncusons; Network model;
Intracellular recordings
1. Introduction
Depth perception is a key feature in vision. I t is used to detect a n d avoid objccts,
and t o maintain distance SI-0111 targets. Animals employ dilfcrent strategies likc
stercopsis, vergence, occlusion, motion parallax and looming; [I-61 to c o m p ~ ~ t e
depth. Most insects, however, have to rely on monocular mechanisms like motion
parallax [7-10) and looming [ I 1-13], because their eyes ase fixed in the head, have
fixed lenses, rind are very close together.
*Corresponding arrthos. Cornput:~tion:II Ncul-obiology I2nho~-atory.
Ilow:~rdt l[rgItcs h4cdic:tl I n s ~ i t u ( c
Salk Institute. 10010 N. TOI-reyPincs Rd. 1~ Jolla CA 92037. USA.
E-11rcri1trcltltws: mar-tina(@saIk.edu ( M . Wicklein).
0925-2312/01/5-sue front mattel- ;('i 2001 Elscvics Scicnce 13.V. All rights scscrvcd
PII: S 0 9 2 5 - 2 3 1 2 ( 0 1 ) 0 0 5 6 6 - 5
Looming, the detection of an apparcnt size increasc in a n approaching object, is
a simple and reliable strategy to measure t11c changc of distance bctwccn a n observcr
and a n object. On its own i t cannot dcfinc a 3-D map of a scenc, but i t can reliably
detect changes in depth. I n contrast to occlusion and motion parallax, looming docs
not involve background features 01-self-motion of tlic obscrver, but only rcquircs that
the object is diflerent from the background to bc detectable. Looming sensitive
ncurons are found tliroughout the animal kingdom including spliingids 1131. grasshoppers [14], pigcons [15], and lnacaquc monkeys 1161.
2. Methods
We used standard intracellular recording techniques with s ~ ~ h s c q u e ndye
t filling
(Neurobiotin) and processing [13], visual stimuli were generated by a software
package (Visionworks) and prcsentcd on n fast computcr monitor (refresh rate of
160 Hz). The models were simulated with custom software using MATLAB.
3. Results
Visual stimuli representing looming or reccding objects can be cliaracterized b y
four parameters: change in luminance; incrcase or decrease of area; increase or
decrease of object perimeter length; and motion of tlie object's perimeter or edge.
Intsacellular recordings reveal visual interncurons in the optic lobes of M. smia tliat
are selectively activated by some of these parameters.
Wicklein and Strausfeld (2000) identified two classes of wide-field neurons that
respond selectively to looming and receding stimuli. The two types of looming
sensitive neurons in M. sexiu use dil'resent mechanisms to detect the approach o r
retreat of an object. They proposed that change of perimeter length may be detected
by class 1 neurons and expansion or contraction visual flowfields by class 2 neurons.
In both cell classes changes in luminance have no role in the detection of looming or
anti-looming and both classes are furthcr subdivided into neurons tliat are excited by
imagc expansion (looming cells) or by image contraction (anti-looming cells).
We created a conceptual model for a "changc In cdge length" dctector and
~mplcmentedtlus model of class 1 ncurons I n MATLAB. T o assess the validity of the
model we tested both the niodel and the neurons w ~ t hthe same stimuli; tlie model
should match the output strength of the ncuron over a w ~ d erange of visual stlmulr
and be consistent with the known anatomy. The stmu11 were presented to the ~ n p u t
laycr of the model, which projects to edge dctcctors whose output was sumtncd
spatially over the whole or parts of tlic ~ n p u tficld, followed by a time dcrivativc. We
M. Wickleit~,T.J. Scjtiowski / Necrrwcotr~pirlbig38-40 (2001) 1595- 1602
1597
of Golgi impl-cgnatcd small-ticld ncurons in the optic lobcs of 1!4s e . ~ / othat a!-c
Fig. I . Reco~lstructio~is
presumed to hc in rhc looming network. lo: lobula. lop: lohula plate. me: ~ncdulla.Tml(w): wide trans~ncdullacclls. Tml(s): small transmedulla cclls. T.4. TS: bushy T-cells. Thc 1x11-intlicatcs 100 p m . Modifcd
after Wicklcin a n d Strausfckl 1131.
used available anatomical data, Golgi studies on small ficld cclls and reconstructions
of type 1 cells [13], to fi~rthcrconstrain the modcl, so that it includcs nlost of the
anatomical properties of the looming-sensitive cell and thc network underlying the
class 1 neurons. Fig. 1 shows rcconstructed small field cclls in the visual system of M.
sevicr that are possible inputs to class 1 looming sensitive ncurons. In Fig. 2, optical
neuropilcs are illustrated with their main layers and small field cclls with their input
Lamina
1
3.k
IE
Lobula P l a t e
I
Medulla
Lobula
Fig. 2. Schematic diagr-am of thc arrangement of small field elcnients in relation to the input arcas o f a type
1 neuron (gray areas). The small field cells proposed as inputs to type 1 cclls arc: Tni I , T4. T5 and Y cclls.
T m l cells have arborizations in the inncrmost riledulla layer and provide inpiit to T5 ncurons, which
pr-ojcct to the lobula plate. Y cells have arborizations in the innermost rnedulla laycr and providc inputs to
both the lobula and the lobula plate. T 4 cells connect the innermost medulla laycr with the lobula plate.
and output regions which we relate to input areas of class 1 looming sensitive cells
(gray areas) as described in Wicklein and Strausf'eld (2000).Class 1 looming sensitive
neurons have inputs in the innermost medulla layer, the outermost lobula layer and
throughout the lobula plate. There are several small field ncuson types that could
serve as inputs to these layers. Considering the architecture and known direction of
information flow through the visual system of insects [17] we can predict in which
order the visual neuropiles receive information. Information IS passed from the retina
to the lamina and the outermost lnedulla layers through short and long visual fibers.
Transmedulla cclls (Tm-cells) then convey the information fsom the outer medulla
to the inner medulla, from there the information flows into the lobula and the lobula
plate via T4, Tm1 o r Y cells. Considering the architecture of the type 1 cells it can be
deduced that the visual information arrives first in the arborizations that reside in the
outermost medulla layer and that there should be a considerable delay before the
same information arrives in the arborizations in the lobula and lobula plate. T h e delay
in information flow could be due to additional cable length required to reach the
M. IVicklcitl. T.J. Scj~~oivski
/ Ncurocon~p~~ting
38-40 (2001) 1595- 1602
Retina
lnterneurons
MedullaT -
TTTA~obula
/
Type 1 cell
Fig. 3. The model consists of an input layer (retina), with retinotopic projections to a layer of edge
detectors. Thc output of each edge detector (ED) is I-elayed by interneurons retinotopically to the inputs of
the type 1 neurons, where they are spatially summed. Each edge detector connects to two interneurons that
relay to the type 1 cell. We propose that the connection to the medulla is direct; the connection to the lobula
and lobula plate is delayed by longer cable or an additional synapse. For a loorning detector the direct
connection to the type 1 neuron would be excitatory. whereas the delayed connection would be inhibitory.
An expanding object excites edge detectors and that excitation is transmitted througli both the excitatory
and inhibitory intcrneul-ons to the type 1 cell. Due to the dill'erent delays in the excitatory and inhibitory
patliway, the information in the inllibitory pathway at time I = 1 coincides with the excitatory information
of time t = 1 + r l at the inputs of the type 1 neuron. This results in a sustained increase in excitation in the
type 1 neuron f01- a n expanding object. A contracting object on the other hand will lead to a decrease of
numbel- of edge detectol-s and therefore the inhibitory delayed input to the type 1 neuron will always be
Sreater than the ciil-cctexcitatory input, thus leading to a decrease in excitation in the type I cell. The open
symbol indicates a n excitatory synapse; the full circle indicates an inhibitory synapse.
lobula and lobula plate (Tml-cells), or an additional synapse that is required (T4, T5
and Y cells) or both. Fig. 3 shows the model for an edge length detector network
including these considerations. We assumed an input area with an edge detector layer
that is connected to two types of interneurons per column, which in turn project onto
the type 1 looming neuron. In the model excitatory connections were made to the
arborizations residing in the medulla, and inhibitory connections to the lobula and
lobula plate arborizations to looming cells. For an anti-looming cell connections to
the lnedulla should be inhibitory and connections to the lobula and lobula plate
[time units]
Fig. 4 . Results from a model simulation of a "looming neuron" plotted for different stimuli. The responses
show a n initial response when the stimulus is presented a n d a stcady state I-esponse.T h e initial responses
are excitating for all the stimuli, which decline to thc sustained levels. The model shows no steady state
response t o moving edges (squares) o r bars (circles). A n expanding object (triangles) leads to a positive
model I-esponse,a contracting object (diamonds) to ;I negative model response. Thc arrow indicates the start
of the stitnulation.
excitatory. The time delay introduced by the architecture of the class I neurons itself
would provide the time delay necessary for the comparison of edge length over time
and therefore the detection of a growing edge (expanding object) versus shrinking edge
(receding object).
The nu~nericaloutput of the inodel is compared to the instantaneous spike frequency of the neuron under the same stimulus condition corrected by the resting spike
frequency. Hence, a zero output represents the resting activity level of the neuron;
a positive output an increased firing activity, a negative output a reduced firing
activity relative to the resting activity level.
Comparing the output of the model simulation with the recordings fsom class 1
neurons (Fig. 4), the model captures many of the essential properties of the physiological data. The model showed strong transient initial responses and sustained steadystate responses that were positive to looming and negative to anti-looming stimuli. No
responses were elicited by moving single bars, edges or gratings (Fig. 4). However, the
model showed a highly pronounced initial response that is not present or at least is
inucl~reduced in the neurons. This could be due to either temporal orland spatial
averaging in the network preceding the neuron or the neuron itself. For objects of
different size the initial responses scaled with edge length but steady-state responses
did not change. The only parameter that influenced the level of steady-state responses
was the rate of expansion/contraction: the responses grow with expansion/
M. Wickleir~.T.J. Srjr~owski/ Newoconlputir~g38-40 (2001) 1595- 1602
160 1
contraction speed. No change is observed for edges, bars or gratings moving with
different velocities.
4. Discussion
The model predicts that class 1 cells should be insensitive to the size of expanding/contracting objects, the spatial frequency and velocity of moving gratings, but be
sensitive to the velocity of expansion or contraction. Invariance of response with
respect to object size would allow the cells to detect change of edge length equally well
for different object sizes and thus allow the animal to hover in front of and forage on
flowers of different corolla sizes. The model predicts that the neuronal response would
code for the rate of expansion/contraction, which in turn would translate into the
approach or retreat speed of the flower. In a behavioral context, a flower that is moved
rapidly toward or away from the moth by a wind gust would elicit a larger response
then a slowly moving flower. To avoid collisions with fast approaching flowers the
moth would have to change its motor output faster and put more force into the
movement. The increased neuronal response might decrease the delay of the motor
response and increase its strength. These predictions regarding different object sizes,
are being tested in experiments
stimulus velocities and rates of expansio~~/contraction
in both cell types by measuring tuning curves for these parameters.
Due to the fixed delays between the medulla input and the lobula/lobula plate
inputs the model should be sensitive to the spatio/temporal properties of the stimulus
similar to motion detector models like the Reichardt detector (see Ref. [18]). Further
experiments will further constrain important model parameters such as the time delay.
Acknowled,uernent s
This work was supported by the NSF and The Howard Hughes Medical Institute.
[I]
[2]
[3]
[4]
[S]
[GI
171
G.J. Andcrsoii. J . Cisncsos, 1'. Atclilcy. A. Saidpour. Spccd. size and cdgo-r;\tc information for
detection of collision events. J. Exp. Psycliol. Hum. Perform. 25 ( 1 ) (1999) 356-260.
T.S. Collctt. Vision: simplc stcrcopsis, Curr. Biol. 6 (1 1 ) (1996) 1302-1395.
G.C. Dc Angclis. Newsomc W T Organisation of disparity-sclcctivc ncurons in macaque area MT.
J. Ncrirosci. 19 (4) ( 1999) 1398- 1415.
H . Ono, B.J. Rogers. M. Olimi. M.E. Ono. Dynamic occlusio~iand motion p:rsnllax in depth
perception, Pcrccpion 17 (2) ( 1988) 255-266.
J.P. Roy, H. Komarsu. R.H. Wurtz. Disparity sensitivity of neurons in monkey cstrxtsiate :~rcaMST.
.I. Ncurosci. 12 (7) ( 1 992) 2478-2392.
1-1. Sun, B.J. Frost. Computation of ciilrcrcnt optical vari:~blcs of looming oljccts in pigeon nucleus
solundus neurons. Nature Ncurosci. 1 (4) (199s) 106-303.
T.S. Collctt. Peering----;\ locust behavior pattern for obtaining motion p;\r;~Il;rxi~ifor~ii;\tion.
J. Esp.
Hiol. 76 (197s) 337-241.
E.C. Sobcl, l>cpth perception by motion parallas and paraclosical 1x11-allasin the locust. N;iturwisscnschaftcn 77 (5) (1990:i) 24 1-243.
E.C. Sobcl. T h e locust's i ~ s cof motion parallas to mcasut-c clistancc. J. Comp. l'hysiol. 107 (5) (I99Ob)
579-588.
M.V. Srinivas;in. How ilisccts infer range from visual ~iiotion.Re\,. Oculomot. Rev. 5 (1993) 139-156.
F.C. Rind. D.I. UI-nmwcll. Neural network based on the input 01-g;inizatic)n of a n idcntilicd ncuron
signaling impending collision. J. Ncul-ophysiol. 75 (1996) 967-985.
F. Gabbiani. I-I.G. Krapp. G . Lnurcnt. Computation of object approach by a widcficld. motionsensitive neuron. J. Neurosci. 19 (3) (1999) 1 172-1 141.
M . Wicklcin. N.J. Strausfcld. The organization ctnd signilicnncz of neurons dctccting change of depth
in the hawk moth Manduca Scstn. J. C o m p . Ncurol. 424(7) (7000) 356-76.
F.C. Rind. Identification of tiircctio~iallysclccti\,e motion-detecting Iicuroncs in thc locust lobula and
their synaptic connections ivitli a n identified descending ncul-on. .I. Esp. 13iol. 149 (1990) 21-43.
Y.C. Wang. 13.3. 1-1-ost.Time to collision is signaled by neurons in tlic I ~ I I C I ~ I I S rotundus of pigcons.
Natul-c 356 (6366) (1992) 736-235.
M.S.A. GI-nzinno. R.A. ili~dcrscn. R.J. Snowden. Tuning of hlST ncul-ons to spiral ~iiotions.
J. Neurosci. 14 ( 1 ) (1094) 54-07.
N.J. Stl-auslld, Atlas of a n I~iscctI3rn1n. Springer. Bcl-lin. Ilcir1clhc1-g.N c ~ vI'ork. 1970.
M. Epelhaaf. A. Uorst. blovcmcnt detection in al-thropods. in: F.A. Miles. J. Wallman (Eds.). Visual
Motion and its Role in Stabiliration of Gaze. Elscvicr. Amsterdam. 1993. pp. 55-77.
Nlartir~aII1icklein earned her P1i.D. in neuroscience at the University of Tiibingen.
Germany. and after a postdoc with Nick Strnusfcld a t the ARL-DN in Tucson.
AZ, she is now a research associate in the C N L at The Salk Institute for Biological
Studies in La Jolla. CA. Her main goal is to understand how animals collect and
compute visual infol-nintion that allows them to orient and behave in coniplcs
t111-cc-dimensional visual environ~iicnts. Her partic~iln~interest lies in feati~res
which are estracted by the visual systcm and how they are used to compute
a hc1iavio1-al response of the animal to stimuli occurring in its natural cnvironment. She is using electsophysiology, computational modcling. behavioral analy.sc.\-/a.
sis. and histochemistry to analyze her model system, the moth il4ot1d~tco
Terrence Scj~io~vski
is a n Investigator with the Howal-d I-lughcs Medical Institute
a n d a Profcssor a t The Salk Institute for 13iological Studies where he directs the
Cornput;~tionalNeurobiology Labol-atory. He is also Professor of Biology at the
University of Califol-nia. San Dicgo. where lie is Director of the Institute for
Ncul-al Computation. Dr. Sejnowski rcccivcd n U.S. in physics from the CaseWestern Rcservc University, an M.A. in physics fl-om 1'1-inccton University, and
a P1i.D. in physics from PI-inceton Univcrsity in 1978. In I9S8. Dr. Scjnowski
fou~itlcd Neural Computation. published by the M I T PI-css. IHe is also the
President of the Neural Information Processing Systems Foundation. T h e longI-;ingcgoal of Dr. Sejnowski's rcscnrch is to build linking principles from brain to
b c I i ; ~ v i ~using
r
computational niodcls. This goal is being pul-sued with a combination of theoretical and experimental :~pproachesa t scvcl.al lcvcls of investigation
ranging from the biophysical level to the systems level.