Rescorla-Wagner Assignment 1
THE RESCORLA-WAGNER MODEL
The Rescorla-Wagner model is a "simple" mathematical model that attempts to simulate changes in the strength of association between CSs and USs.
Mathematical modeling is a way of developing precise theoretical explanations.
A good model of some natural process (e.g., conditioning a dog to salivate when a bell rings) will not only simulate the facts already documented, but will also suggest circumstances and outcomes that have not yet been observed. Our confidence in the model will grow if, when the suggested circumstances are experimentally created, the results predicted by the mathematical model actually occur. On the other hand, if the predictions of the model are not confirmed, then we know that the learning process inside the animal is different from that of the particular mathematical model, and that we must either propose a different mathematical model or seek an explanation in some other medium.
Mathematical modeling has been used extensively in theoretical psychology.
One of the most impressive models in the field of associative learning is one proposed by Robert Rescorla and Allan Wagner. The Rescorla-Wagner model is impressive for three main reasons: a) it accounts for most of the basic facts of conditioning, b) it suggests novel circumstances and outcomes (most of which are downright counterintuitive), and c) it is mathematically elegant (simple). For these reasons, the model has been a major export to other areas of psychology
(social cognition, human judgment about causes and effects) in situations in which an associative account of the data is desirable.
The model simulates changes in the strength of a learned association between a CS and US. This is what learning curves presumably depict. Early in conditioning the association is weak and, correspondingly, CR strength is low.
As CS-US pairings accumulate, more learning takes place and the association grows stronger. CRs occur more often and are of greater magnitude. Ultimately, when learning is complete, the CS-US association reaches its final high level and a vigorous CR is observed on every trial.
The Rescorla-Wagner model describes a learning mechanism by which the strength of associative connections is adjusted on a trial-by-trial basis. The model learns "episodically" as a result of contiguity between events. Thus, it is a contiguity-learning device in the tradition of Aristotle and Pavlov.
However, unique characteristics of the model result in its giving rise to contingency-learning effects of the kind originally thought to be irreconcilable with a contiguity mechanism. The model consists of the following formula:
Δ
V =
αβ
(
λ
- VALL), where
V= the current associative strength of a given CS. This corresponds to the current ability of a particular CS to evoke a CR (positive value = excitation, negative value = inhibition).
Δ
V = the change in V (increment or decrement) on a given trial. For example, if a previously reinforced CS is extinguished by presenting it without the US, the CS's associative strength (its strength of connection with the US) will diminish. The result is negative value for
Δ
V.
λ
= the maximum associative strength the US is capable of supporting. Generally speaking, as US intensity increases, so does
λ
. Thus, a US of zero intensity
(i.e., no US) has
λ
= 0.

Rescorla-Wagner Assignment 2
VALL = the sum of the associative strengths (V values) of all CSs present on a given trial. For example, if CSA, CSB, and CSC are presented simultaneously in compound (i.e., ABC), then VALL = VA + VB + VC. On the other hand, VALL for compound AB is simply VA + VB. If CSB is present, then VALL = VB. The quantity
VALL represents what the subject expects on a given trial, based on the CSs present. The associative strength of absent CSs does not change, nor does it contribute to VALL.
(
λ
- VALL) = The difference between the US obtained on a trial and that expected on the basis of the CSs present. In most cases, this difference is the
"balance" which remains to be learned.
α
= a rate-learning parameter for a given CS. This parameter is an index of the "salience" of the CS. Although for any particular CS
α
is fixed, its value may differ across CSs from 0 to 1.0. A loud banging noise that is very
"salient" would have a relatively high
α
(e.g., .25). A barely perceptible noise will have a relatively low
α
(e.g., .0025).
β
= a rate of learning parameter for the US. A potent US will have a value near
1.0. This may vary from 0 to 1.0.
APPLICATIONS OF THE MODEL
To facilitate reading (not to mention typing!), instead of the symbols
CSA, etc., we will identify events with letter names. In the following simulations, there are four CSs with the labels A (
α
= .25, light), B (
α
= .25, tone), C (
α
= .1, vibration), and X (
α
= .25, noise). We assume that
β
= 1.0, and that
λ
= 100 when the US is presented and
λ
= 0 when the US is omitted.
Let's create a table for each application so we can "track" changes in the various components of the model on a trial-by-trial basis. In the first example, a light (A) and vibration (C) occur on alternate trials and are always reinforced with the US (+). The last two columns of the table allow us to watch the growth of the A-US and C-US associations. Each CS's strength only changes on trials in which it is actually presented and reinforced with the US. We assume that learning is permanent and there is no forgetting between trials.
λ
VALL
Δ
VC VA VC
1 A+ 100 25 0
2 C+ 100 10 25 10
3 A+ 100
4 C+ 100
5 A+ 100 43.75 14.06 --- 57.81 19
6 C+ 100 19 --- 8.1 57.81
7 A+ 100 57.81 10.55 --- 68.36 27.1
8 C+ 100 27.1
9 A+ 100 68.36
10 C+ 100
As you can see the learning process is faster for A than C. This makes sense because A has a greater "salience" than C. Notice, the greatest changes occur on early trials.

Rescorla-Wagner Assignment 3
The greatest strength of the Rescorla-Wagner model has been the successes it has seen in compound conditioning experiments. Consider how the model explains Kamin's blocking effect. In blocking, one CS is first trained as a strong excitor (it predicts the US). A second, redundant CS, then joins the original CS and the two CSs together continue to signal the US. Lets denote this procedure as A+ then AX+ to reflect these two stages of training. Kamin found little conditioning to the added CS, X. Can the Rescorla-Wagner model simulate blocking?
Group A+ then AX+
λ
VALL
Δ
VX VA VX
1 A+ 100 25 0
2 A+ 100
3
4
A+ 100 43.75 14.06 --- 57.81 0
A+ 100 57.81 10.55 --- 68.36 0
5 A+ 100 68.36
6 AX+ 100 76.27
7 AX+ 100 88.13
8 AX+ 100 94.07
9 AX+ 100 97.03 11.12
10 AX+ 100 98.51 11.49
Note, X has acquired very little associative strength by the tenth trial,
VX = 11.49. A few pages from now we discover that if 5 AX+ trials had occurred without the prior conditioning of A, then X's strength would have been considerably higher, i.e., VX = 48.44. Thus, in this simulation, A can be said to have blocked learning to X. Thus, according to the model, CSs only acquire associative strength if they provide new information about the US. Conditioning does not proceed every time the CS and US are paired. Here, X is paired with the US but is not strongly associated with it because the US is already well predicted by A. If we had trained A until its associative strength was 100 before introducing the AX compound trials, X would not have acquired any associative strength at all. The blocking was incomplete in this example because A had not reached its final level (i.e., VA was not 100).
Pavlov found that a particular type of compound training procedure allowed a stimulus to become a conditioned inhibitor (it signals an expected US would not occur). Using A and B to represent two CSs, lets see what the model predicts. In this simulation A is the "excitor" and B is the "inhibitor".
Group A+, AB- (trials alternated)
λ
VALL
Δ
VB VA VB
1 A+ 100 25 0
2
3
4
5
AB- 0 25 -6.25 -6.25 18.75 -6.25
A+ 100 18.75 20.31 --- 39.06 -6.25
AB- 0 32.81 -8.20 -8.20 30.86 -14.45
A+ 100 30.86 17.29 --- 48.15 -14.45
6
7
8
9
AB-
A+ 100 39.72 15.07 --- 54.79 -22.88
AB-
A+
0
0
100
10 AB- 0
33.70 -8.43 -8.43 39.72 -22.88
31.91 -7.98 -7.98 46.82 -30.86
46.82 13.30 --- 60.12 -30.86
29.26 -7.32 -7.32 52.8 -38.18

Rescorla-Wagner Assignment 4
Eventually, after many more trials than just simulated, we would find that
VA = 100 and VB = -100. Thus, the model simulates Pavlov's basic conditioned inhibition finding. In this situation, B is a conditioned inhibitor. It carries negative associative strength and cancels A's excitation when the two are presented in combination.
Now, consider a three group experiment in rabbit eye-blink conditioning where each group receives the compound AX (e.g., light and noise presented simultaneously) and paired with a US (e.g., air puff to the eye). These AX compound trials alternate with another kind of trial. For Group X WEAK, alternate trials consist of stimulus X presented alone (i.e., without being followed by the US). For Group X STRONG, alternate trials consist of X paired with the US. For Group CONTROL, these alternate trials are empty; that is, no
CS or US occurs. Thus, all groups have the same experience with respect to the
AX compound and, particularly, with respect to stimulus A and its contiguity with the US. Therefore, common sense or "old contiguity theory" might well lead us to expect all groups to show the same level of conditioned responding to A after, say, five pairings of AB with the US. When an experiment of this design is actually conducted, however, the groups differ dramatically with respect to their conditioned responsiveness to A. The Rescorla-Wagner model provides us with an account.
Group X WEAK (AX+, X-)
λ
VALL
Δ
VX VA VX
1 AX+ 0 25 25 25 25
2 X- 0 25 --- 25 18.75
3
4
AX+
X-
100
0
43.75 14.06 14.06 39.06 32.81
32.81 --- -8.20 39.06 24.61
5 AX+ 100 63.67
6 X- 0 33.69 --- -8.42 48.14 25.27
7 AX+ 100 73.41
8 X- 0 31.92 --- -7.98 54.79 23.94
9 AX+ 100 78.73
10 X- 0 29.26 --- -7.31 60.11 21.95
Group X STRONG (AX+, X+)
λ
VALL
Δ
VX VA VX
1 AX+ 0 25 25 25 25
2 X+ 100
3 AX+ 100 68.75
4 X+ 100 51.56 --- 12.11 32.81 63.67
5 AX+ 100 96.48 64.55
6 X+ 100 64.55
7 AX+ 100 107.1 -1.78 -1.78 31.91 71.63
8 X+ 100 71.63
9 AX+ 100 110.6 -2.66 -2.66 29.25 76.06
10 X+ 100

Rescorla-Wagner Assignment 5
Group CONTROL (AX+)
λ
VALL
Δ
VX VA VX
1 AX+ 0 25 25 25 25
3 AX+ 100 50 12.5 12.5 37.5 37.5
5 AX+ 100 75 6.25
7 AX+ 100 87.5 3.13 3.13 46.88
9 AX+ 100 93.76
The expected results of a test with the A alone after 10 trials follow:
Group CONTROL = 48.44, Group X WEAK = 60.11, and Group X STRONG = 29.25.
Although the numbers we have used are strictly hypothetical, the expected results for A are quite interesting. Response strength in the presence of A differs markedly depending on the treatment of X: If X is strongly associated with the US (Group X STRONG), less associative strength is acquired by A than if
X is weakly associated with the US (Group X WEAK). The control group in which X was not presented outside of the AX compound lies in the middle. This demonstrates a principle called the competition hypothesis : CSs compete for the level of associative strength supported by the US, a result confirmed in an experiment conducted by Wagner and Saavedra (Wagner, 1969).
The preceding simulation makes it easy to see how the Rescorla-Wagner model, which assumes a contiguity mechanism, can account for contingency learning, if one includes the incidental "background" or "contextual" stimuli that are inevitably present during learning. For example, a tone (A) that is paired with a fear-evoking shock produces makes a rat afraid of the tone. But this pairing occurs in a particular context--a distinct experimental chamber
(X). Thus, the surrounding contextual cues of the chamber are also paired with shock. If the shock does not occur in the absence of the tone (context), then a perfect positive contingency would exist (
Δ
P = +1.0). This situation resembles the events experienced by Group X WEAK. The context is not reinforced during the intertrial interval (X-) and the tone and context are reinforced together
(AX+). In this case, the model predicts a short-lived increase in responding to the context (X) followed by a decline to zero. The tone becomes strongly excitatory. Alternatively, if extra shocks are presented in the intertrial interval, this would reduce the magnitude of the positive contingency (i.e.,
Δ
P is less than +1.0) and conditioning to the tone should decline. This situation resembles the simulation of Group X STRONG. (i.e., the context is reinforced during the intertrial interval and this undermines conditioning to the tone.
The important thing to appreciate is that we have an explanation of contingency learning via a contiguity process. Isn't that neat!

Rescorla-Wagner Assignment
Assume:
Beginning Associative Strengths: VA = 100, VB = -100, VC = 0, VD = 50.
Example 1: A is followed by a medium US for 1 trial:
Parameter Values:
α
A =
α
B =
α
C =
α
D = 0.25,
β
= 1.0, and
λ
= 100 (big
US), 50 (medium US), or 0 (no US)
(+100)
λ
VALL VA VA
1 A+ 50 100 -12.5 87.5 because
Δ
VA =
αΑ β
(
λ
- VALL) = 0.25 X 1.0 (50 – 100) = -12.5 and thus, new VA = oldVA +
Δ
VA = 100 + (-12.5) = 87.5
Example 2: AD is followed by no US for 1 trial:
λ
(+100)
VALL VA
Δ
VD VA VD
1 AD- 0 150 -37.5 -37.5 62.5 12.5 because VALL = VA + VD = 100 + 50 = 150
Δ
VA =
αΑ β
(
λ
- VALL) = 0.25 X 1.0 (0 – 150) = -37.5
Δ
VD =
α
D
β
(
λ
- VALL) = 0.25 X 1.0 (0 – 150) = -37.5 and thus, new VA = oldVA +
Δ
VA = 100 + (-37.5) = 62.5 new VD = oldVD +
Δ
VD = 50 + (-37.5) = 12.5
6

Rescorla-Wagner Assignment 7
For All Questions Assume:
Beginning Associative Strengths: VA = 100, VB = -100, VC = 0, VD = 100.
Parameter Values:
α
A =
α
B =
α
C =
α
D = 0.25,
β
= 1.0, and
λ
= 100 (big
US), 50 (medium US), or 0 (no US)
1) Create a table that shows what is expected to happen if C is paired with a big US for 10 trials (i.e., C+ trials,
λ
= 100). (1 point)
2) Create a table that shows what should happen if C is paired with the medium
US for 10 trials (i.e., C+ trials,
λ
= 50). (.50 point) Compare the results with this table with the table you created for Question #1. (.25 points) Does the
Rescorla-Wagner model correctly predict the influence of US magnitude on conditioning? (.25 points)
3) Create a table that shows what will happen to the associative value of A (VA
= 100) if it is followed by “no US” for 10 trials. (1 point) Have experiments shown that excitors lose positive strength when they are followed by no US? (.25 points) . Create a table that shows what will happen to the associative value of
B (VB = -100) if it is followed by “no US” for 10 trials. (1 point) Have experiments shown that conditioned inhibitors lose negative strength when they are followed by no US? (.25 points)
4) Assume the BC combination is reinforced with the big US (i.e., BC+). Model the changes in associative strength of these two CSs during 10 consecutive BC+ trials. (1 point) Compare the trial-by-trial strengths of VC in this question with the results you obtained in Question 1. To do this, draw a line graph with trials on the X-axis and strength, Vc, on the Y-axis. There should be two lines representing the strength of C, one line for Question #1 and one line for
Question #4 (.25 points ). What is the name of the phenomenon shown in the graph?
(.25 points )
5) According to the Rescorla-Wagner model, what changes should occur in the associative strengths of A and D if they are presented in compound and reinforced for 10 trials with a big US (i.e., AD+)? (1 point) What is the name of this phenomenon where a loss of associative value occurs despite pairings with the US? (Hint: see Page 88) (.25 points)
6) Complete the following table (1 point ) Do you think the discrimination will eventually be learned (Vall = 0 on AD trials, and Vall = 100 on each of the A and D trials)? Why or why not?
(.5 points )
λ
VALL
Δ
VD VA VD
1 AD- 0 ____ ____ ____ ____ ____
2 A+ 100 ____ ____ ____ ____ ____
3 D+ 100 ____ ____ ____ ____ ____
4 AD- 0 ____ ____ ____ ____ ____
5 A+ 100 ____ ____ ____ ____ ____
6 D+ 100 ____ ____ ____ ____ ____
7 AD- 0 ____ ____ ____ ____ ____
8 A+ 100 ____ ____ ____ ____ ____
9 D+ 100 ____ ____ ____ ____ ____

Rescorla-Wagner Assignment 8
7) Model 10 trials in which the AB compound is reinforced with a big US. (1 point) Does VA become abnormally strong with strength greater than 100.
What is the name of this phenomenon? (.25 points )

Rescorla-Wagner Assignment 9