The Role of Prior Knowledge in
Human Reconstructive Memory
Mark Steyvers
Pernille Hemmer
University of California, Irvine
1
Interaction between Prior Knowledge & Episodic Memory
reconstructive memory
Episodic Memory
Prior Knowledge,
Semantic Memory,
“Schemas”
“What objects do you remember from your hotel room?”
“What coffee did you order from Starbucks last week?”
2
Research questions

Machine learning


Psychology



How can we formalize prior knowledge (“schemas”)?
How do humans integrate memory and prior knowledge?
What errors do people make?
Interface between human & machine learning


Can we explain human error with rational inference principles?
Can we build better information retrieval systems by taking
cognitive processes into account?
3
Remembering Objects from a Graduate Office
chair
desk
skull
books
(30% of subjects)
Brewer & Treyens (1981)
Experiments



perception: list objects you see in a scene
memory: list objects you remember from a scene
guessing: list objects you might see in a kitchen/office/…
Dining
Hotel
Kitchen
5
Human Data: Precision vs. Output Position
1
Precision
Cumulative Accuracy
0.95
10 Secs.
Time = 10secs
Time = 2secs
Scene Cue
0.9
0.85
0.8
2 Secs
0.75
0.7
0.65
0.6
0.55
2
4
6
8
10
12
14
16
Output Position
6
Bayesian Analysis of Reconstructive Memory
p(objects | memory )  p(memory | objects ) p(objects )
Posterior
Likelihood
Prior
What objects were
studied given noisy
memory contents?
How likely is this
memory content
given these objects
were studied?
How likely are
these objects a
priori?
7
Problem 1: Infer prior knowledge
Prior
Knowledge
kitchen1
kitchen2
?
kitchen3
Hierarchical Beta processes (Thibaux & Jordan, 2007);
related to Indian Buffet process
8
Hierarchical Beta Process
(Thibaux & Jordan, 2007)
“Kitchen”
General
knowledge
b
b ~ Beta(c0b0 , c0 (1  b0 ))
Object probs.
for specific
image
aj
Reported
objects
participants
a j ~ Beta( c j b, c j (1  b))
y
yij ~ Bernoulli( a j )
objects
Kitchen65
Kitchen45
9
Problem 2: Recall using Prior + Episodic M.
Prior
Knowledge
b
semantic memory
Actual objects
in image
y
?
episodic memory
Noisy Memory
Content
x
10
Data
Model
1
1 Time = 10secs
Time = 2secs
0.95 Scene Cue
10 Secs.
Cumulative Accuracy
Precision
Cumulative Accuracy
0.95
0.9
0.85
0.8
2 Secs
0.75
0.7
0.65
0.6
0.55
2
4
6
8
10
12
Output Position
14
16
0.9
0.85
0.8
0.75
0.7
0.65
=0.200
0.6
=0.150
=0.000
0.55
2
4
prior only
6
8
10
12
14
16
Output Position
11
Limitations of Model

Independence assumption between objects

Recall often clusters correlated objects

“computer”  “mouse”  “keyboard”
12
Reconstructing the Style of Written Digits
Study
Digit space from “Earth Mover” distances
personal
prior
Reconstructed
reconstructed
studied digit
13
Reconstructing Drawings from Memory
Studied Drawing
Reconstruction
14
Reconstructing Lists of Words
Study this list:
PEAS, CARROTS, BEANS, SPINACH,
LETTUCE, HAMMER, TOMATOES,
CORN, CABBAGE, SQUASH
HAMMER,
PEAS,
CARROTS,
...
15
Conclusion

Prior knowledge has strong effect on memory

Guessing based on prior knowledge leads to quite good
performance

Machine learning methods help formalize “schemas”

Useful to understand human memory when designing
information retrieval systems
16
Thanks!
17