Memory across Eye-Movements:
1/f Dynamic in Visual Search
Deborah J. Aks Gregory Zelinsky Julien C. Sprott
UW-Whitewater
SUNY- Stonybrook
Aug 5, 2001
Madison, Wisconsin
----------------------------------
Society for Chaos Theory in
Psychology & the Life Sciences
UW-Madison
What guides eye-movements during complicated
visual search?
• Memory?
• Are there correlations across sequence of
fixations?
• Deterministic rules?
• Simple set of neuronal interaction rules (e.g.,
SOC) ?
Find:
Do we keep track of where we look?
Is there memory in search?
Horowitz, T.S. & Wolfe, J. M. (1998). Visual
Search has no memory. Nature, 357, 575-577.
Finding: Random repositioning of stimuli
does not affect search RTs
No memory?
Overview
•
QUESTIONS.
• What guides complicated eye movements?
• Random or non-random process?
• Is there memory across fixations?
•
METHOD OF TESTING.
• Challenging visual search task
•
KEY ANALYSES
• Power law relation?
• Coloring of noise --> Memory across eye-movements
• Fourier analysis
• Iterated Functions Systems (IFS) Test
•
RESULTS
• Raw fixations --> short term memory (1/f2 brown noise)
• Fixation differences --> long term memory (1/f pink noise)
•
MODEL. Self-organized criticality (SOC)
(Bak, Tang, & Wiesenfeld, 1987)
•
CONCLUSION
• There is memory across eye-movements!
• SOC model predicts relative eye movements.
Horowitz, T.S. & Wolfe, J. M. (1998). Visual
Search has no memory. Nature, 357, 575-577.
Finding: Random repositioning of stimuli
does not affect search RTs
Key Press RTs
vs.
Eye Movements
What does visual search teach us?
•
Cognitive processes!
– Speed & Accuracy
– Mechanisms
• Automatic or Attention
– Search strategy
• Parallel, Serial, random or…?
Features...
•
•
Find the odd item
Discriminate by..
– Color
xxxxxxx
– Size
x xxxxx
– Orientation
------l---
x
– Depth
– Movement
xxxx-->
x
Look for the red L
L
L
L
L
L
L
L
L
L
L
L
L
L
L
Conjunction Search
•
Find...combination of features
– 2 orientations (particular arrangement)
• Find: L among Ts
T
T
L
T
T
T
T
T
T
T
T
T
T
T
T
T
L
T
+
T
T
T
T
T
T
L
T
T
T
T
T
T
Feature search is easy!
RT (msec)
–Fast (300ms)
–Parallel (0-10ms/item)
–No attention needed
500
400
300
0 ms/item
5
10
# of items
15
Conjunction search is hard!
–Slower
–Sequential
–Focused attention needed
RT (msec)
Conjunction
40 ms/item
700
500
Feature
0 ms/item
300
5
10
# of items
15
•
Feature search is easy
– Fast (300ms)
– Parallel (<10ms/item)
– No attention needed
•
Conjunction search is difficult
– Slow (>500ms)
– Serial (>10ms/item)
– Attention needed
What guides search?
•
Environmental information.
•
Internal cognitive process
• Attention.
• Memory?
• Deterministic Process:
Self-Organized Criticality (SOC)?
Memory in visual search?
Evidence for…
We are able to keep track of
where we look!
Inhibition of return
Evidence against
Failure to replicate
inhibition of return
(Wolfe & Pokorny, 1990)
(Klein, 1982)
– Memory for locations in
search (Kristjansso,2000)
– Identity of objects
accumulates over time
(Treisman & Gelade, 1980)
– Random repositioning of
stimuli does not affect
search RTs
(Horowitz & Wolfe, 1998)
– Inattentional amnesia in
search (Wolfe, 1999)
Non-systematic eye-movements
Engle, 1977; Ellis & Stark, 1988;
Scinto & Pillalamarri, 1986;
Krendel & Wodinsky, 1960; Groner & Groner, 1982
Visual Search Task
Find the upright “T”
T
T
T
T
T
T
T
T
T
T
T
T
T
T
T
T
T
T
T
T
T
T
T
T
T
T
T
T
T
T
T
T
T
T
T
Method.
• Each trial contained 81 Ts.
• 400 trials lasting 2.5 hours.
• 1 second central fixation
• Eight 20-minute sessions separated by 5-minute rest
• Generation V dual purkinje-image (DPI) tracker
Map trajectory of eyes:
• Duration & x,y coordinates for each fixation.
---------------------------------------------------------• Differences between fixations
xn – xn+1 & yn – yn+1
•
Distance = (x2 + y2)1/2
•
Direction = Arctan (y/x).
Analyses
• Descriptive & Correlational Statistics
• Power spectra (FFT)
• Iterated Function Systems (IFS) test
Results
• 24 fixations per trial (on average)
• 7.6 seconds (SD =6.9 sec) per trial (316 ms/item).
• Mean fixation duration = 212 ms (SD = 89 ms)
• 10,215 fixations across complete search experiment.
Series of Fixation Differences
(yn+1- yn)
Fixations
Scatter plot ofEye
10,215
eye fixations for
the entire visual search experiment.
Delay Plot of Fixations
yn -vs- y n+1
Across 8 sessions we see scaling properties:
•Fixation frequency decreased from 1888 to 657
•Fixation duration increased from 206 to 217 ms.
• Fixation differences…
• xn – xn+1 decreased
• yn – yn+1 increased
Spectral analysis
Fast-Fourier Transform (FFT)
Power vs. Frequency
Regression slope = power exponent
fa
f -2 = 1/ f 2
Brown noise
Power law indicates…
• Adaptive fractal properties:
• Scale invariance
• Flexible system
• Strength of memory
• Steepness of the slope (on a log-log scale) reflects..
• correlation across data points
• duration of memory
1/f 0 noise -- flat spectrum= no
correlation across data points
1/f noise --shallow slope =
extremely long term correlation
1/f 2 noise-- steep slope =
short-term correlation.
White Noise
Pink Noise
Brown Noise
Power Spectra on raw fixations
a
Power Spectra of first differences
across fixations
a = -.6
Distance across eye fixations
(x2 + y2) 1/2
a = -.47
a = -0.3
a = -1.8
Iterated Function Systems
--IFS Test-(Peak & Frame, 1994; Stewart, 1989).
Fixation Series
2
3
Start
1
4
2
3
1
4
2
3
1
4
2
3
1
4
2
3
1
4
2
3
1
4
2
3
1
4
2
3
1
4
2
3
1
4
2
3
1
4
2
3
1
4
2
3
1
4
2
3
1
4
2
3
1
4
2
3
1
4
2
3
1
4
2
3
1
4
2
3
1
4
2
3
1
4
2
3
1
4
2
3
1
4
2
3
1
4
2
3
1
4
2
3
1
4
2
3
1
4
White Noise
1/f 
PinkNoise
1/f 
Brown Noise
1/f 
Raw Fixations
Clustering along diagonals reveals short-term, highly correlated consecutive data points
Brown noise pattern
Fixation differences
Triangular microstructure associated w/ long-term, loosely correlated consecutive data points
Pink noise
IFS Test: Fixation direction
Clustering indicates correlated fixations. Direction of fixations show
anti-correlated movements a indicated by absence of main diagonals.
IFS tests yields patterns
consistent w/ FFT results
Summary of results:
• Sequence of…
• Absolute eye positions --> 1/f brown noise
– Short-term memory.
• Differences-between-fixations --> 1/f pink noise
– Longer-term memory.
Model
Hebb, 1969; Rummelhardt & McClelland, 1985
Neuronal interactions --->
implicit guidance
Could eye movements be described by a simple set of
neuronal interaction rules (e.g., SOC) that produce 1/f
behavior?
SOC Network
(Adapted from Bak, Tang, & Wiesenfeld, 1987)
0
4
Increasing Neural Activation --->
•Stimulate 1 neuron
Z(x,y)= initially
stimulated site
Threshold rule: For Z(x,y) > Zcr =3
As individual neurons are activated beyond a threshold (of 3), activity (4) is
dispersed to surrounding cells.
Z(x,y) -> Z(x,y) - 4
Activity in the
original site is
depleted to zero.
Z(x,y)-> Z(x,y) + 1
Surrounding activity
increases by 1
4
0
Neural SOC …
Neural SOC w/ eye movements trails
Eye movements are pulled to the
site(s) of greatest activation
{
• For Z(x,y) > Zcr
• Z(x,y) -> Z(x,y) - 4
• Z(x + 1,y)-> Z(x + 1,y) + 1
• Z(x,y + 1) -> Z(x,y + 1) + 1
Simple set of SOC rules..
…can produce:
• Complex & effective search
CONCLUSIONS
• There is memory across eye-movements!
• Neural SOC model --> 1/f relative eye-movements.
• Simple self-organizing system--> effective search
Aks, D. J. Zelinsky G. & Sprott J. C. (2002). Memory Across EyeMovements: 1/f Dynamic in Visual Search. Nonlinear
Dynamics, Psychology and Life Sciences, 6 (1).
http://psychology.uww.edu/Aks/papers/AZS01.ppt
Bluebird contributed by
www.Sierra foothill.org
And thanks to Bob Goodman for,
among a # of things,
getting me to reduce the # of slides in this talk.
Phew, Deb’s down to 93 slides
References
Aks, D.J., Nokes, T. Sprott, J.C. & Keane, E. (1998). Resolving
perceptual ambiguity in the Necker Cube: A dynamical systems
approach. Abstracts of the Psychonomics Society, 3, 38.
Allport, D.A. (1987). Selection-for-action: Some behavioral and
neurophysiological considerations of attention and action. In H.
Heuer & A.F. Sanders, (Eds.), Perspectives on Perception and
Action. Hillsdale, N.J.: Erlbaum.
Bak, P. (1996). How nature works: the science of self-organized
criticality. New York: Springer-Verlag.
Bak, P., & Tang, C. (1989). Earthquakes as a self-organized
critical phenomenon. Journal of Geophysics Research - Solar.
Earth Planet, 94, 15635-15637.
Bak, P., Tang, C., & Wiesenfeld, K. (1987). Self-organized
criticality: An explanation of 1/f noise. Physical Review
Letters, 59, 381-384.
Bak, P., Tang, C., & Wiesenfeld, K. (1988). Self-organized
criticality. Physical Review A, 38, 364-374.
Daugman, J. G. (1991). Self-similar oriented wavelet pyramids:
Conjectures about neural non-orthogonality. In A. Gorea (Ed.),
Representations of Vision. Cambridge: Cambridge University Press.
Ellis, S. R., & Stark, L. (1986).
Statistical dependency in visual
scanning. Human Factors, 28(4), 421-438.
Gilden, D. L. (1996). Fluctuations in the time required for
elementary decisions. Psychological Science, 8 (4), 296–302.
Gilden, D. L., Thornton, T., & Mallon, M. (1995). 1/f noise in
human cognition. Science, 267, 1837-1839.
Hilborn, R. C. (1994). Chaos and Nonlinear Dynamics: An
introduction for Scientists and Engineers. Oxford: Oxford
University Press.
Hochberg, J. (1968). In the minds eye. In R. N. Haber (Ed.),
Contemporary theory and research in visual perception. (pp. 309331). New York: Holt, Reinhart & Winston.
Horowitz, T.S. & Wolfe, J. M. (1998). Visual Search has no memory.
Nature, 357, 575-577.
Humphreys, G. W., Quinlan, P. T., & Riddoch, M. J. (1989). Grouping