Supplementary Materials for:
Integrating Formal and Grounded Representations in
Combinatorics Learning
David W. Braithwaite and Robert L. Goldstone
Indiana University
INTEGRATING REPRESENTATIONS IN COMBINATORICS
2
Section A. Problem sets used for pretest and posttest.
In each problem set, problems 1-2 represent near transfer problems, problems 3-4 far transfer problems,
and problems 5-6 very far transfer problems. The near and far transfer problems were included in both
Experiments 1-2, while the very far transfer problems were only included in Experiment 2.
Problem Set 1
1. "Suppose a farmer is designing a crop rotation plan. He will rotate among different crops over a period
of several years. The farmer plans to rotate 4 different crops: Potatoes, Turnips, Beans, and Lettuce. Each
crop will be used for exactly 1 year over a 4-year period. For example, one possible sequence is: LettuceTurnips-Beans-Potatoes. How many difference sequences are possible that use each of the 4 crops exactly
once?”
2. "Again, suppose a farmer is designing a crop rotation plan. He will rotate among different crops over a
period of several years. This time, suppose the farmer plans to rotate 6 different crops: Beans, Corn,
Eggplant, Lettuce, Potatoes, and Turnips. Each crop will be used for exactly 1 year over a 6-year period.
For example, one possible sequence is: Corn-Turnips-Eggplant-Lettuce-Beans-Potatoes. How many
difference sequences are possible that use each of the 6 crops exactly once?"
3. "Again, suppose a farmer is designing a crop rotation plan. He will rotate among different crops over a
period of several years. This time, suppose the farmer plans to rotate either 4 different crops (Potatoes,
Turnips, Beans, and Lettuce), 5 different crops (the same 4 crops plus Corn), or 6 different crops
(the same 5 crops plus Eggplant). Whichever crop set he chooses, each crop in the set will be used for
exactly one year. For example, the following are all possible sequences: Lettuce-Turnips-Beans-Potatoes,
Lettuce-Corn-Turnips-Beans-Potatoes,
Lettuce-Corn-Turnips-Eggplant-Beans-Potatoes.
How
many
difference sequences are possible that use one of the above crop sets and use each crop in the set exactly
once?"
INTEGRATING REPRESENTATIONS IN COMBINATORICS
3
4. "Again, suppose a farmer is designing a crop rotation plan. He will rotate among different crops over a
period of several years. This time, suppose the farmer plans to rotate 4 from among these 6 crops Beans, Corn, Eggplant, Lettuce, Potatoes, and Turnips. Whichever 4 crops he chooses, each crop in
the set will be used exactly for exactly one year over a 4-year period. For example, the following are both
possible
sequences:
Turnips-Beans-Potatoes-Corn,
Corn-Eggplant-Potatoes-Lettuce.
How
many
difference sequences are possible that use 4 out the 6 crops, and use each of those 4 exactly once?"
5. "Next, suppose the farmer is choosing crops to plant in several different fields for one year. There are
3 fields altogether, and for each field, the farmer must choose one of the 6 crops. (It's possible to choose
the same crop for more than one field, or even all 3 fields.) In how many different ways can the farmer
choose crops for his fields?"
6. "Finally, suppose that the farmer decides to test out how well each of his crops grows when using
different kinds of fertilizer. He has 6 different crops and 4 types of fertilizer. He plans to test out each
possible combination of crop with fertilizer. How many different combinations will he test out?"
Problem Set 2
1. "Suppose Jim is getting married. He must select his groomsmen and decide the order in which they will
stand (far left, 2nd from left, 3rd from left, etc.). Jim has decided to select his 4 closest (male) friends as
groomsmen: Dave, Ed, Steve, and Tom. But he still must decide the order in which they will stand. For
example, one possible arrangement is: Tom-Dave-Steve-Ed. How many different arrangements are
possible for the 4 groomsmen?"
2. "Again, suppose Jim is getting married. He must select his groomsmen and decide the order in which
they will stand (far left, 2nd from left, 3rd from left, etc.). This time, suppose Jim has decided to select his
6 closest (male) friends as groomsmen: Dave, Ed, George, Mike, Steve, and Tom. But he still must
decide the order in which they will stand. For example, one possible arrangement is: Mike-George-TomDave-Steve-Ed. How many different arrangements are possible for the 6 groomsmen?"
INTEGRATING REPRESENTATIONS IN COMBINATORICS
4
3. "Again, suppose Jim is getting married. He must select his groomsmen and decide the order in which
they will stand (far left, 2nd from left, 3rd from left, etc.). This time, suppose that Jim is considering to
select either his 4 closest friends (Dave, Ed, Steve, and Tom), his 5 closest friends (the same 4 plus
George), or his 6 closest friends (the same 5 plus Mike). Whichever group of friends he chooses, he
also must decide the order in which they will stand. For example, the following are all possible
arrangements: Tom-Dave-Steve-Ed, Mike-Tom-Dave-Steve-Ed, Mike-George-Tom-Dave-Steve-Ed. How
many different arrangements are possible that use either his 4 closest friends, his 5 closest friends, or his 6
closest friends?"
4. "Again, suppose Jim is getting married. He must select his groomsmen and decide the order in which
they will stand (far left, 2nd from left, 3rd from left, etc.). This time, suppose that Jim is considering to
select 4 from among his 6 closest (male) friends: Dave, Ed, George, Mike, Steve, and Tom.
Whichever 4 friends he chooses, he also must decide the order in which they will stand. For example, the
following are both possible arrangements: Tom-Dave-Steve-Ed, Ed-George-Steve-Mike. How many
different arrangements are possible that use 4 from among his 6 closest friends?"
5. "Next, suppose Jim is going to ask his groomsmen to help with some tasks before the wedding. There
are 3 tasks altogether (reserving a limo, printing programs, and buying liquor) and for each task, he must
choose one of his 6 groomsmen. (It's possible to choose the same groomsman for more than one task, or
even all 3 tasks.) In how many different ways can he assign groomsmen to tasks?"
6. "Finally, suppose that Jim wants to take some wedding photos together with his groomsmen as well as
his closest female friends. He has 6 groomsmen and 4 female friends. Each photo will include one
groomsman and one female friend, and he will take one photo for each possible combination. How many
photos will he take altogether?"
INTEGRATING REPRESENTATIONS IN COMBINATORICS
5
Section B: Problems used as examples during training.
1. "Suppose three horses - Amber, Beryl, and Crystal - run in a race. Assuming all three finish the race,
how many different arrangements of first, second, and third place are possible? For example, one possible
arrangement is Beryl first, then Amber, then Crystal. "
2. "Suppose four horses - Amber, Beryl, Crystal, and Diamond - run in a race. Assuming all four finish
the race, how many different arrangements of first, second, third, and fourth place are possible? For
example, one possible arrangement is Beryl first, then Amber, then Diamond, then Crystal. "
3. "Suppose a traveller wants to visit three major cities in Europe - London, Paris, and Rome - and is
considering in which order to visit them. Assuming she visits each city exactly once, how many different
orders are possible? For example, one possible order is Paris first, then London, then Rome. "
4. "Suppose a traveller wants to visit four major cities in Europe - Berlin, London, Paris, and Rome and is considering in which order to visit them. Assuming she visits each city exactly once, how many
different orders are possible? For example, one possible order is Paris first, then London, then Berlin, then
Rome. "
INTEGRATING REPRESENTATIONS IN COMBINATORICS
6
Section C. Text of and performance on comprehension questions (Experiment 1).
After completing the training section, but before beginning the posttest, participants in
Experiment 1 answered four multiple choice comprehension questions. All four questions referred to two
quantities, (A) and (B), and had the same set of possible responses, namely “(A) is larger than (B),” “(B)
is larger than (A),” “(A) and (B) are equal,” and “Cannot be determined from the information given.”
The four questions and their correct answers were as follows:

Question A. “Suppose a group of people comes to a restaurant and each person chooses a main
course. Compare the number of possible outcomes in two cases: (A) if each person chooses a
different main course from what the others choose, and (B) if each person chooses independently
(i.e. could be either the same as or different from the others).” Answer: (B) is larger.

Question B. “Suppose a group of people comes to a restaurant and each person chooses a main
course. Compare the number of possible outcomes in two cases: (A) if one additional person joins
the group and chooses a main course, and (B) if two additional people join the group, and both
choose the SAME main course.” Answer: (A) and (B) are equal.

Question C. “Suppose a certain restaurant offers meal sets consisting of one appetizer and one
main course. Compare the number of possible meal sets in two cases: (A) if there are 5 options
for appetizer and 5 options for main course, and (B) if there are 6 options for appetizer and 4
options for main course.” Answer: (A) is larger.

Question D. “Suppose a certain restaurant offers meal sets. Compare the number of possible meal
sets in two cases: (A) if the meal sets consist of one appetizer, one main course, and one drink,
with 2 options for each, and (B) if the meal sets consist of one appetizer with 2 options and one
main course with 5 options.” Answer: (B) is larger.
INTEGRATING REPRESENTATIONS IN COMBINATORICS
7
The proportions of correct responses overall and by training condition are shown in the table below:
Question
Total Correct
Pure Listing
Pure Formalism
List Fading
Formalism First
Question A
54%
63%
52%
39%
62%
Question B
30%
37%
33%
14%
34%
Question C
55%
44%
59%
54%
62%
Question D
82%
89%
85%
75%
79%
Proportions of correct responses ranged from 30% to 82%. Question B was the most challenging,
with only 30% of participants answering correctly. This question required an understanding that the
number of possible outcomes for two events that are constrained to have the same outcome is the same as
the number of possibilities for one event alone. Participants who did not give the correct answer were
equally likely to think that either number of possibilities was larger. Questions A and C were also
challenging for participants, with only slightly over half answering each correctly. Question A required
an understanding that a constraint on the possible outcomes would necessarily decrease their number, so
that (B) would be larger than (A). Besides the correct answer, participants were about equally likely to
say that (A) was larger or that the answer could not be determined.
Questions C and D could both be solved by using the fundamental principle of counting,
described in the Introduction. This principle was implicit in the instruction participants received, but was
not explained explicitly. Interestingly, while these two questions appear to involve the same principle,
participants answered Question D correctly (82%) much more often than Question C (55%). (The most
common incorrect answer for Question C was to say that (B) was larger than (A).) The reasons for this
difference cannot be determined based on the data collected, because participants were not asked to show
work for these questions. It is possible that a correct answer was easier to obtain for Question D than for
Question C via simple, though incorrect, heuristics, e.g. some participants may have noticed that, of the
two options in Question D, (A) involved a single number (2) while (B) involved two numbers (2 and 5),
and thus judged (B) to be larger.
INTEGRATING REPRESENTATIONS IN COMBINATORICS
8
Because the pretest did not include comprehension questions, it is impossible to determine
whether participants’ comprehension of combinatorics actually improved due to the instruction they
received. However, in order to determine whether the different training conditions had a differential
effect on comprehension, the total number of comprehension questions answered correctly was entered
into an ANCOVA with training condition as a between-subjects factor and pretest score as a covariate.
The effect of condition was not significant, F(3,106)=1.73, p=0.164, indicating that participants’
performance on the comprehension questions was not differentially affected by training condition.
INTEGRATING REPRESENTATIONS IN COMBINATORICS
9
Section D: Standards for coding shown work on test and training problems.
1. Numerical Calculation. This code was used if numbers and some kind of operator appeared in
participant’s work, such as *, +, ^, !, forming an arithmetic expression. This code was also used if the
English words for any of these operators appeared instead of the symbols, as in "five times four", "three
factorial", etc..
2. Exponent. Only applied if Code 1 also applied. This code was used if participant used an exponential
expression, e.g. "two to the third power," "2 to the 3rd power," "2^3". The code was also used in the case
of repeated multiplication of the SAME number by itself AT LEAST 3 TIMES, e.g. "3*3*3,” but not if
the number only appeared twice, e.g. "3*3").
3. Factorial. Only applied if Code 1 also applied. This code was used if participant used a factorial
expression, e.g. "three factorial,” "3 factorial," "3!". The code was also used in the case of repeated
multiplication of a number by successively smaller numbers, e.g. "3*2*1. " The code was applied if the
number "1” was omitted from such a sequence, but not if any other number was omitted, e.g. the code did
not apply to "5*4*3".
4. Outcome Listing. Participant listed at least two outcomes, at least one of which did not appear in the
problem statement. This listing could be done using letters, words, or anything else to represent specific
sampling events.
5. Verbal Reasoning. This code applied when participants’ work included worse or phrases in English
which did not just repeat information in the problem statement, and also were not just an English
language way of saying one of the things covered by the preceding codes. For example, "four crops”
would not count (repeating problem statement), "five times four” also would not count (this would get
Code 1 instead), but "four crops for each” would count because of the words "for each,” and "four
crops=24, five crops=5*24” would count because the language is being used to explain the calculations
(not just repeating the problem statement).
INTEGRATING REPRESENTATIONS IN COMBINATORICS
10
6. Other. This code was mainly used for solution methods that did not fall into any of the other codes
(including the Code 7).
The most common use of this code was for visual diagrams or pseudo-
arithmetical expressions such as "__ * __ * __ = __,” which apparently were used to conceptualize the
relations between successive sampling events.
7. No Work Shown. This code was used if the answer space was empty (e.g. only whitespace characters
were entered), merely repeated information from the problem statement, or only contained a number
without any other explanation or calculation. This code was exclusive with all other codes.
INTEGRATING REPRESENTATIONS IN COMBINATORICS
11
Section E: Detail on training materials used in Experiment 2.
Four types of standard solutions were created for each problem: formalism-based, listing-based,
mixed/listing-first, and mixed/formula-first. Only the solutions were displayed on screen, while the
explanations of the solutions were presented through an accompanying audio track.
The solution
representations were built up incrementally, as in Experiment 1. Screenshots of the final, complete
representations from each type of standard solution are shown below.
Final screen from formalism-based standard solution to training problem 1.
INTEGRATING REPRESENTATIONS IN COMBINATORICS
Final screen from listing-based standard solution to training problem 1.
Final screen from mixed standard solutions to training problem 2.
12
INTEGRATING REPRESENTATIONS IN COMBINATORICS
Sample training videos for each solution type are available online at the following locations:
Formalism-based solution for training problem 1:
http://www.youtube.com/watch?v=xzEO7JYaMb0
Listing-based solution for training problem 1:
http://www.youtube.com/watch?v=qnKuFPM7vnQ
Mixed/formalism first solution for training problem 2:
http://www.youtube.com/watch?v=g_AZMjVBt3s
Mixed/listing first solution for training problem 2:
http://www.youtube.com/watch?v=VG82wMUaBB4
13
INTEGRATING REPRESENTATIONS IN COMBINATORICS
14
Section F. Analysis of Effects of Pretest Strategy Use on Transfer Performance in Experiment 2.
Participants in Experiment 2 were classified as having used factorials and/or outcome listing on
pretest just as in Experiment 1. For consistency, only near and far transfer problems were used for these
classifications. Pretest factorial use and listing use were then added as factors to the linear mixed model
described earlier to analyze the effects of training condition and transfer distance on transfer performance.
Neither the main effect of factorial use nor that of listing use was significant, F(1,83.46)=.384, p=.537 for
factorial use and F(1,64.16)=.528, p=.470 for listing use, nor was their interaction, F(1,63.97)=1.09,
p=.300. Importantly, no significant interactions were found between training condition and listing use,
factorial use, or both, ps>.40. The effects of distance and pretest score were consistent with those
reported elsewhere in the paper and thus are not repeated here. A significant 3-way interaction was found
between factorial use, listing use, and transfer distance, F(2,131.85)=3.30, p=.040. Apart from this
interaction, whose complexity renders its interpretation unclear, the two training conditions do not appear
to have differentially affected transfer performance among participants with different strategy use
tendencies.