Precis:
Learning psychologists have long been interested in how animals and humans integrate their associative, reward, and motivational experiences into a final choice. This seminar will review both historical and contemporary attempts to resolving the issue from Thorndike (1898) to Dickinson & Balleine (in press). In doing so, we shall contrast a variety of meta-theoretical and theoretical approaches seeking empirically assessable contrasts where possible. The concepts and experiments will range from basic animal learning paradigms to drug abuse to neuroscience mechanisms of behavior.
Seminar meetings will focus on a major reading each week and participants will take turns as seminar leaders. Leaders will read an additional paper or two as part of the preparation.
[ 11 meetings only. Not: Sept 6, Oct 4, Nov 8, Dec 13]
General References:
Hilgard, Ernest R. (Ed). (1978). American psychology in historical perspective.. American
Psychological Association. viii, 558 pp. (available on line at APA/PsycArticles)
Benjamin, Ludy T. Jr. (Ed). (1988). A history of psychology: Original sources and contemporary research . Mcgraw-Hill Book Company. xiii, 560 pp.
Reference suggestions for weekly topics:
1. The Past….
Warren, Howard C. (1916). Mental association from Plato to Hume.
Psychological Review.
23(3), May 1916, 208-230.

Hume, David (1711-1766): A Treatise of Human Nature.
In Benj. Rand, (Ed). (1912). The classical psychologists: Selections illustrating psychology from Anaxagoras to Wundt . (pp. 279-
312). Houghton, Mifflin and Company. xxii, 734 pp. http://content.apa.org/books/10885-020.pdf?sid=60e8dd04-a5bb-4450-a549-ec9497666e9f
Bricke, John. (1974). Hume's associationist psychology. Journal of the History of the
Behavioral Sciences. 10(4), Oct 1974, 397-409.
2. The Eastern Europeans: Pavlov, Bekterev, and Konorski: Brain models of associations
Pavlov. I.P. (1927/1960). Lecture II. Conditioned Reflexes . (Pp. 16-32). Dover
Konorski, J. (1948). An outline ofPavlov’s theory of activity of the cerebral cortex. In J
Konorski, Conditioned Reflexes and Neuron Organization
University Press.
. (pp 7-33). Cambridge
Bekhterev, V.M. (1932). General Principles of Human Reflexology , trans. 4th Russian ed. New
York: International Publishers. ( Also 1932 in London; originally published as Objective
Psychology in 1907).
Doty, RW (2006). Konorski and conditional reflexes: A historical summary and an addendum.
Acta Neurobiologiae Experimentalis , 66, 91-97.
Miller, S.; Konorski, J. (1969/1928). On a particular form of conditioned reflex.
Journal of the Experimental Analysis of Behavior . 12(1), 187-18
Konorski, J.; Miller, S. (1937). On two types of conditioned reflex.
Journal of General Psychology. 16, 264-272.
.
Konorski, J. (1964). Some problems concerning the mechanism of instrumental condition ing
Acta Biologiae Experimentalis.
24(2), 59-72.
3. Thorndike Reinforced Associations
Thorndike, E. L. (1898) "Animal Intelligence: an Experimental Study of the Associative
Processes in Animals," Psychological Review Monograph Supplement,
Whole No. 8 (1-109) [ An exerpt : http://content.apa.org/journals/amp/53/10/1125.pdf
)
2 (1898): No. 4.
Thorndike, E.L. (1913) The Psychology of Learning (Educational Psychology, II).
New York:
Teacher’s College. Pp. 1-31

Hull, Clark L. (1935). Thorndike's Fundamentals of Learning.
Psychological Bulletin.
32(10), Dec 1935, 807-823.
………………..
Hull, Clark L. (1937). Mind, mechanism, and adaptive behavior. Psychological Review , 44, 1-
32. [Also in Hilgard, Ernest R. (Ed). (1978). perspective
American psychology in historical
. (pp. 309-336). American Psychological Association. viii, http://content.apa.org/books/10049-017.pdf?sid=6ffd62e3-55d3-40a3-bc94-ab072657e70a ]
.
Hull, Clark L.(1950). Behavior postulates and corollaries--1949.
Psychological Review . 57(3), 173-180.
4. Wagner-Rescorla Instantiation
Rescorla, R.A., & Wagner. A.R. (1972). A theory of Pavlovian conditioning: Variations in effectiveness of reinforcement and non-reinforcement. In A.H. Black & W.F. Prokasy
(Eds.), Classical Conditioning II: Current Research and Theory
Century Crofts.
. (Pp 64-98). Appleton
Wagner, A.R. ( 1971). Elementary associations. In HH Kendler & JT Spence (eds), Essays in
Neo-Behaviorism.
(Pp 187-213). Appleton Century Crofts.
ALSO: http://www2.psy.uq.edu.au/~landcp/PY269/r-wmodel/r-wmodel.html
Miller, R R.; Barnet, R C.; Grahame, N J. (1995). Assessment of the Rescorla-Wagner model.
Psychological Bulletin . 117(3), May 1995, 363-386.
5 . Tolman and re-introduction of motivations and goals as objects as learned
Tolman, E.C. (1932/ 1949/1967). Purposive Behavior in Animals and Men . Appleton Century
Crofts. [While a key reference and worth reading, Tolman is likely better summarized by others than presented here by himself. Sample…]
Tolman, Edward Chace (1938). The determiners of behavior at a choice point . Psychological
Review, 45, 1-41. [Also in Hilgard, Ernest R. (Ed). (1978). American psychology in historical perspective. (pp. 337-370). American Psychological Association. viii, 558 pp. http://content.apa.org/books/10049-018.pdf?sid=3c84e18e-c58b-4e04-b3ca-e8684ff221fd ]

Bolles R.C. (1972) Reinforcement, expectancy, and learning. Psychological Review , 79, 394-
409. [Bolles was a Tolman student.]
6. Mowrer: Motivations as learned mediating ACTIVATORS in Two-Process theory.
Mowrer, O.H. (1947). On the dual nature of learning—A reinterpretation of “conditioning” and
“problem solving”.
Harvard Educational Review , 17, 102-148.
Rescorla R.A. & Solomon, R.L. (1967). Two-process learning theory: Relationships between
Pavlovian conditioning and instrumental learning. Psychological Review , 74, 151-182.
Konorski, J. (1964). Some problems concerning the mechanism of instrumental conditioning
Acta Biologiae. Experimentalis , 24, 59-72
Zielinski, K. (1985). Jerzy Konorski’s thory of conditioned reflexes.
Acta Neurobiologiae
Experimentalis, 45, 173-186.
7. Stimulus

Outcome Expectations as CUES for responses
Hull, C.L. (1931). Goal attraction and directing ideas conceived of as habit phenomena
Psychological Review.
38(6), Nov 1931, 487-506.
Trapold, MA, & Overmier, JB. (1972) The second learning process in instrumental learning. In
AH Black & WF Prokasy (eds.), Classical Conditioning II: Current Theory and
Research.
(Pp 427-452). Appleton Century Crofts.
Overmier, JB, & Lawry, JA. (1979). Pavlovian conditioning and the mediation of behavior. In G.
Bower (ed.), The Psychology of Learning and Motivation, 13, 1-55.
Overmier, J.B., Bull, J.A., III, & Trapold, M.A. (1971). Discriminative cue properties of different fears and their role in response selection in dogs. Journal of Comparative &
Physiological Psychology , 76, 478-482.
Other…

Kruse, John M.; Overmier, J. Bruce; Konz, Wilbert A.; Rokke, Eric (1983). Pavlovian conditioned stimulus effects upon instrumental choice behavior are reinforcer specific.
Learning and Motivation. 14(2), 165-181.
Peterson, Gail B.; Trapold, Milton A. (1980). Effects of altering outcome expectancies on pigeons' delayed conditional discrimination performance. Learning and Motivation.
11(3), 267-288.
Colwill, Ruth M.; Rescorla, Robert A. (1988). Associations between the discriminative stimulus and the reinforcer in instrumental learning.
Journal of Experimental Psychology: Animal
Behavior Processes.
14(2), Apr 1988, 155-164.
BUT , the challenge
Rescorla, Robert A. (1994) . Control of instrumental performance by Pavlovian and instrumental stimuli.
Journal of Experimental Psychology: Animal Behavior Processes . 20(1), Jan
1994, 44-50.
Therefore:
8. Response

Outcome associations in Two-Process response control.
Colwill, Ruth M. (1994). Associative representations of instrumental contingencies.
In
Medin, Douglas L. (Ed), The psychology of learning and motivation: Advances in research and theory, Vol. 31.
(pp. 1-72). Academic Press.
Colwill, Ruth M.; Rescorla, Robert A (1990).
Effect of reinforcer devaluation on discriminative control of instrumental behavior.
Journal of Experimental Psychology: Animal Behavior
Processes.
16(1), Jan 1990, 40-47.
Colwill, Ruth M.; Rescorla, Robert A. (1990 ). Evidence for the hierarchical structure of instrumental learning . Animal Learning & Behavior. 18(1), Feb 1990, 71-82.
Rescorla, Robert A.; Colwill, Ruth M. (1989 ). Associations with anticipated and obtained outcomes in instrumental learning. Animal Learning & Behavior . 17(3), Aug 1989, 291-
303.
Rescorla, Robert A. (1990 ). Instrumental responses become associated with reinforcers that differ in one feature.
Animal Learning & Behavior . 18(2), 206-211.
Rescorla, Robert A. (1990). The role of information about the response^outcome relation in instrumental discrimination learning.
Behavior Processes.
Journal of Experimental Psychology: Animal
16(3), Jul 1990, 262-270.

Rescorla, Robert A. (1990). Evidence for an association between the discriminative stimulus and the response^outcome association in instrumental learning. Journal of Experimental
Psychology: Animal Behavior Processes.
16(4), Oct 1990, 326-334.
9. Motivational Control of Associations with Outcomes
Dickinson, Anthony; Nicholas, D. J. (1983). Irrelevant incentive learning during instrumental conditioning: The role of the drive-reinforcer and response-reinforcer relationships. The
Quarterly Journal of Experimental Psychology B: Comparative and Physiological
Psychology.
35B(3), 249-263.
Dickinson, Anthony; Dawson, G. R. (1988). Motivational control of instrumental performance:
The role of prior experience of the reinforcer . The Quarterly Journal of Experimental
Psychology B: Comparative and Physiological Psychology . 40(2-B), 113-134.
Balleine, Bernard; Dickinson, Anthony. (1991 ). Instrumental performance following reinforcer devaluation depends upon incentive learning.
The Quarterly Journal of Experimental
Psychology B: Comparative and Physiological Psychology.
43B(3), 279-296.
Dickinson, Anthony; Balleine, Bernard . (1995).
Motivational control of instrumental action.
Current Directions in Psychological Science . 4(5), , 162-167.
OR
Dickinson, Anthony; Balleine, Bernard (1994). Motivational control of goal-directed action.
Animal Learning & Behavior.
22(1), 1-18.
Dickinson, Anthony; Balleine, Bernard (2002).
The role of learning in the operation of motivational systems.
In H. Pashler & C.R. Gallistel (Ed ). Steven's handbook of experimental psychology (3rd ed.), Vol. 3: Learning, motivation, and emotion.
533). John Wiley & Sons, Inc. x, 900 pp.
(pp. 497-
Lopez, Matias; Balleine, Bernard; Dickinson, Anthony. (1992).
Incentive learning following reinforcer devaluation is not conditional upon the motivational state during re-exposure.
The Quarterly Journal of Experimental Psychology B: Comparative and Physiological
Psychology . 45B(4), 265-284.

Dickinson, Anthony; de Wit, Sanne. (2003). The interaction between discriminative stimuli and outcomes during instrumental learning.
The Quarterly Journal of Experimental
Psychology B: Comparative and Physiological Psychology.
56B(1), 127-139.
Balleine, Bernard W.; Paredes-Olay, Concepción; Dickinson, Anthony (2005). Effects of
Outcome Devaluation on the Performance of a Heterogeneous Instrumental Chain.
International Journal of Comparative Psychology . 18(4), 2005, 257-272.
And just how critical details can be (trying to reconcile discrepant results):
Balleine, Bernard; Dickinson, Anthony (1992). Signalling and incentive processes in instrumental reinforcer devaluation. The Quarterly Journal of Experimental Psychology
B: Comparative and Physiological Psychology . 45B(4), Nov 1992, 285-301.
10. Computational Causal Learning
Shanks, David R.; Dickinson, Anthony. (1987). Associative accounts of causality judgment .
Bower, Gordon H. (Ed). The psychology of learning and motivation: Advances in research and theory, Vol. 21. (pp. 229-261). Academic Press. vii, 319 pp.
Dickinson, Anthony (2001). Causal learning: Association versus computation.
Current Directions in Psychological Science . 10(4), 127-132.
OR
Dickinson, Anthony (2001). Causal learning: An associative analysis.
The Quarterly Journal of Experimental Psychology B: Comparative and Physiological
Psychology . 54B(1), 3-25.
Dickinson, Anthony; Balleine, Bernard W. (2000). Causal cognition and goal-directed action.
In C. Heyes & L. Huber, Ludwig (Ed ). The evolution of cognition., Vienna series in theoretical biology . (pp. 185-204). The MIT Press. viii, 386 pp.
[Are they arguing the same as Rescorla or something more than an association??]
Shanks, David R.; Lopez, Francisco J.; Darby, Richard J.; Dickinson, Anthony . (1996).
Distinguishing associative and probabilistic contrast theories of human contingency judgment.
In Shanks, David R. (Ed); Holyoak, Keith (Ed); Medin, Douglas L. (Ed).
Causal learning . (pp. 265-311). Academic Press. xii, 435 pp.

OTHER TOPICS WE COULD EXPLORE:
Human Causal learning of contingency:
Wasserman, Edward A.; Chatlosh, D. L.; Neunaber, D. J. (1983). Perception of causal relations in humans: Factors affecting judgments of response-outcome contingencies under freeoperant procedures
Or
. Learning and Motivation . 14(4), Nov 1983, 406-432.
Wasserman, E. A.; Elek, S. M.; Chatlosh, D. L.; Baker, A. G. (1993). Rating causal relations:
Role of probability in judgments of response^outcome contingency . Journal of
Experimental Psychology: Learning, Memory, and Cognition.
19(1), 174-188
Shanks, David R.; Dickinson, Anthony. (1987). Associative accounts of causality judgment.
Bower, Gordon H. (Ed). The psychology of learning and motivation: Advances in research and theory, Vol. 21 . (pp. 229-261). Academic Press.
Cheng, PW. 1993). Separating causal laws from causal facts. Pressing the limits of statistical relevance. In D. Medin (ed.), Psychology of Learning and Motivation, v 30 . (Pp )
Academic Press.
Cheng, PW. (1997). From covariation to causation: A causal Power theory. Psychological
Review, 104, 367-405.
Gallistel’s new(er) theory:
Gallistel, C. R.; Gibbon, John (2000). Time, rate, and conditioning.
Psychological Review.
107(2), 289-344.
Gallistel, C. R.; Gibbon, John (2001). Computational versus associative models of simple conditioning. Current Directions in Psychological Science . 10(4), 146-150.
Gallistel, C. R. (2002). Conception, perception and the control of action.
Trends in Cognitive Sciences.
6(12), Dec 2002, 504.
Balsam, Peter D.; Fairhurst, Stephen; Gallistel, Charles R. (2006). Pavlovian Contingencies and
Temporal Information. Journal of Experimental Psychology: Animal Behavior
Processes . 32(3), Jul 2006, 284-294.