Psychology 209: Models of Cognitive Processes
Winter, 2012-2013
Syllabus
Key:
†
*
~
_
+
!
Foundational material you should know – read if you don’t already know this
Essential readings for homework or in-class discussion
Readings very closely related to lecture, adding more detail, for reference
Other readings, sometimes historical
Models we won’t cover, possible starting places for projects
Quantitatively challenging materials, sometimes very long, for those interested
Feb 20: Simple Recurrent Networks: Finding Structure in Time
* Elman, J.L. (1990). Finding Structure in Time. Cognitive Science 14, 179-211. A
seminal article introducing an alternative approach to time and syntax.
~ Servan-Schreiber, D., Cleeremans, A., and McClelland, J.L. (1991). Graded State
Machines: The Representation of Temporal Contingencies in Simple Recurrent
Networks. Machine Learning 7, 161-193. Builds on Elman (1990) and provides the basis
for the exercises for the SRN
+ McClelland, J.L. (in preparation). Explorations in Parallel-Distributed Processing: A
Handbook of models, programs, and exercises, Second Edition. Chapter 7: Simple
Recurrent Networks.
_ Elman, J.L. (1993). Learning and development in neural networks: the importance of
starting small. Cognition, 48, 71-99. This and the next article address an interesting issue
but come to very different conclusions. A side topic for those interested.
_ Rohde, D.L.T. and Plaut, D.C. (1999). Language acquisition in the absence of explicit
negative evidence: how important is starting small? Cognition, 72, 67-109.
Feb 22: Deep Belief Networks
* Hinton, G. E. and Salakhutdinov, R. R.(2006). Reducing the dimensionality of data
with neural networks. Science, Vol. 313. no. 5786, pp. 504 - 507.
_ Ranzato, M. Boureau, Y-L, and LeCun, Y. (2007). Sparse feature learning for deep
belief networks, Advances in Neural Information Processing Systems (NIPS 2007).
_ Bengio, Y., Lamblin, P., Popovici, D. & Larochelle, H. (2006) Greedy layer-wise
training of deep networks. Technical Report, 1282, Univ of Montreal.
Feb 25: Learning in Recurrent Networks
One-Page Project Proposals Due
* Williams, R. J. and Zipser, D. (1995). Gradient-based learning algorithms for recurrent
networks and their computational complexity. In: Y. Chauvin and D. E. Rumelhart (Eds.)
Back-propagation: Theory, Architectures and Applications, Hillsdale, NJ: Erlbaum.
~ Movellan, J. R., & McClelland, J. L. (1993). Learning continuous probability
distributions with symmetric diffusion networks. Cognitive Science, 17, 463-496.
~ O'Reilly, R.C. (1996). Biologically Plausible Error-driven Learning using Local
Activation Differences: The Generalized Recirculation Algorithm. Neural Computation,
8, 895-938.
+ McClelland, J.L. (in preparation). Explorations in Parallel-Distributed Processing: A
Handbook of models, programs, and exercises, Second Edition. Chapter 8: Recurrent
Backpropagation.
Feb 27: Reinforcement Learning
*/+ McClelland, J.L. (in preparation). Explorations in Parallel-Distributed Processing: A
Handbook of models, programs, and exercises, Second Edition. Chapter 9: Temporal
Difference Learning.
Mar 1: Efficient Sensory Coding
* Olshausen, B. A. and Field, D. (2004). Sparse coding of sensory inputs. Current
Opinion in Neurobiology, 14:481-487.
~ Smith, E. C. & Lewicki, M. J. (2006). Efficient auditory coding. Nature, 439, 978-982.
~ Karklin, Y., and Lewicki, M. S. Emergence of complex cell properties by learning to
generalize in natural scenes. Nature, 457, 83-86.
Mar 4: Dynamics of Decision Making
* Bogacz,R., Usher, M., Zhang, J. & McClelland, J. L. (2012). Extending a biologically
inspired model of choice: multi-alternatives, nonlinearity and value-based
multidimensional choice. In A. K. Seth, T. J. Prescott and J. J. Bryson (Eds.), Modelling
Natural Action Selection. pp 91-119. Cambridge, UK: Cambridge University Press.
~ Gold, J. I. and Shadlen, M. N. The Neural Basis of Decision Making. Annual Review
of Neuroscience, 30, 535-574
! Wong, K.F. and Wang, X.J. A recurrent network mechanism of time integration in
perceptual decisions. Journal of Neuroscience, 26, 1314-1328. Very interesting because
it links from detailed neuronal models to behavior. The on-line supplement is also
available in the class readings directory for those who are interested.
! Bogacz, R., Brown, E., Moehlis, J., Holmes, P., & Cohen, J.D. (2006). The physics of
optimal decision making: A formal analysis of models of performance in two-alternative
forced choice tasks. Psychological Review, 113, 700-65. This is a very technical paper
and is provided only for those interested.
Feb 6: Executive functions and attention
* Cohen, J. D., Dunbar, K. and McClelland, J. L. (1990). On the Control of Automatic
Processes: A Parallel-Distributed Processing Account of the Stroop Effect. Psychological
Review, 97, 332-361.
~ Botvinick, M. and Plaut, D. C. (2004). Doing without schema hierarchies: A recurrent
connectionist approach to normal and impaired routine sequential action. Psychological
Review, 111, 395-429.
~ Moody, S. L., Wise, S. P., di Pellegrino, G., & Zipser, D. (1998). A model that
accounts for activity in primate frontal cortex during a delayed matching-to-sample task.
The Journal of Neuroscience, 18 (1), 399-410.
_ Cohen, J. D., Servan-Schreiber, D., & McClelland, J. L. (1992). A parallel distributed
processing approach to automaticity. American Journal of Psychology, 105, 239-269.
_ Miller, E. K. and Cohen, J. D. (2001). An Integrative Theory of Prefrontal Cortex
Function. Annual Review of Neuroscience, 24, 167-202.
March 8: The Binding Problem and Complementary Object processing systems in
Dorsal and Ventral Cortex
Lecture by Cynthia Henderson
* Henderson, C. M. & McClelland, J. L. (2011). A PDP model of the simultaneous
perception of multiple objects. Connection Science, 23, 161-172.
Readings to be determined
Mar 11, 13, 15: Project Presentations
Mar 20, 5:00 pm. Final Project Due
Topics Not Covered
Topic A: Connectionist Approaches to Language Processing
* Elman, J. L. (2009). On the meaning of words and dinosaur bones: Lexical knowledge
without a lexicon. Cognitive Science. 33(4), 547–582.
~ McClelland, J.L., St. John, M., and Taraban, R. (1989). Sentence Comprehension: A
Parallel Distributed Processing Approach. Language and Cognitive Processes, 4, 287335.
_ Rohde, D.L.T. (1999). A Connectionist Model of Sentence Comprehension and
Production. Unpublished PhD thesis proposal, School of Computer Science, Carnegie
Mellon University, Pittsburgh, PA.
_ Tabor, W., Juliano, C., & Tanenhaus, M. K. (1997). Parsing in a dynamical system: An
attractor-based account of the interaction of lexical and structural constraints in sentence
processing. Language and Cognitive Processes, 12, 211-271.
Topic B: Disorders of Lexical and Semantic Processing
* Dilkina, K., McClelland, J. L. & Plaut, D. C. (2008). A single-system account of
semantic and lexical deficits in five semantic dementia patients. Cognitive
Neuropsychology, 25(2), 136-164.
_ Rogers, T. T., Lambon Ralph, M. A., Garrard, P., Bozeat, S., McClelland, J. L.,
Hodges, J. R., and Patterson, K. (2004). The structure and deterioration of semantic
memory: A neuropsychological and computational investigation. Psychological Review,
111, 205-235.
_ Plaut, D. C. and Shallice, T. (1993). Deep dyslexia: A case study of connectionist
neuropsychology. Cognitive Neuropsychology, 10, 377-500.
_ Plaut, D. C., McClelland, J. L., Seidenberg, M. S., and Patterson, K. (1996).
Understanding normal and impaired word reading: Computational principles in quasiregular domains. Psychological Review, 103, 56-115.