CREATIVITY AND PROBLEM SOLVING SKILLS
AS A FUNCTION OF LEARNING TRANSFER
by
Benjamin Saltsman
Submitted to the System Design and Management Program
in Partial Fulfillment of the Requirements for the Degree of
Master of Science in Engineering and Business Management
at the
Massachusetts Institute of Technology
February 2002
( Benjamin Saltsman. All rights reserved.
The author hereby grants to MIT and Ford Motor Company permission to reproduce and to
distribute publicly paper and electronic copies of this document in whole or in part.
Signatures of Author
Benjamin Saltsman
Certified by
Of
Dan Ariely
Thesis Supervisor
Associate Professor, Sloan School of Management, MIT
Accepted by
GM LFM Professo
Steven D. Eppinger
Co-Director, LFM/SDM
Co-Director, CIPD
Management Science and Engineering Systems
Accepted by
Paul A. Lagace
Co-Director, LFM/SDM
Professor of Aeronautics & Astronautics and Engineering Systems
MASSACHUSETTS INSTITUTE
OF TECHNOLOGY
JUL 1 8 2002
LIBRARIES
ACKNOWLEDGMENTS
The author wishes to thank Ford Motor Company for giving him the opportunity
to be part of this exciting program at MIT thus fulfilling a long-time dream. This
extraordinary learning experience has given me precious insights and allowed to
make new friends.
The author wishes to thank his thesis advisor, Professor Dan Ariely, for
suggesting this interesting topic, and his invaluable guidance and assistance
throughout this project.
The author would like to thank his classmates for providing a challenging,
stimulating and competitive environment and setting sky-high standards.
I would like to thank the SDM staff for their hard work and dedication.
Last, but not least, I would like to express my profound gratitude to my parents
for instilling in me the values of good education and providing the moral and
operational support throughout this journey.
Page 2 of 2
ABSTRACT
Processes of learning and the transfer of learning are central to understanding
how people develop important competencies. Since early childhood people are
exposed to various types of learning experiences: instruction, tutoring, selfdiscovery, etc.
Knowledge and skills acquired through these various types of
experiences lead to varying levels of proficiency.
The focus of this thesis is to answer the question which type of learning
experience not only provides adequate learning, but also positively affects
learning transfer, defined as the ability to extend what has been learned in one
context to new contexts.
This positive learning transfer is the foundation of
effective problem solving skills highly sought out in today's environment. While
the topic of learning transfer is discussed extensively in the literature, the link
between learning transfer on the one hand and creativity and problem solving
ability on the other hand remains largely unexplored.
Experiment was conducted in which the data was analyzed from 84 engineers,
students and professionals. These individuals were randomly assigned to one of
the three groups. Each of the groups received varying levels of instructions and
asked to solve the same set of puzzles.
The respondents were measured on
several parameters (speed, correctness, etc.). The results of this study show that
while the instructions help narrow the scope of the solution space by focusing the
effort and steering the respondents away from the erroneous directions, if the
instructions do not fit the problem formulation well, or are not transparent to the
respondent, they become a liability. Instructions stifled creativity in this
experiment and adversely affected the problem solving skills of the respondents.
Page 3 of 3
TABLE OF CONTENTS
CHAPTER 1.......................................................................................................8
CREATIVITY...............................................................................................
8
DEFINING CREATIVITY .........................................................................
8
LEVELS OF INVENTION ..................................................................................
9
Two MODELS OF INNOVATION AND INVENTION: INDIVIDUAL VS CULTURAL
........................................................................................................................
CREATIVITY PROCESS.....................................................................................
Can people be taught to be more creative?..........................................
StructuredCreativity Techniques ........................................................
TRIZ Overview ................................................................................
Structured Inventive Thinking Overview..........................................
Summary of Structured Creativity Techniques.................................
Creativity Barrier...................................................................................
How to reduce the creativity barrier .................................................
10
11
11
12
12
14
16
17
21
CHAPTER 2.................................................................................................22
TRANSFER OF LEARNING.....................................................................22
WHAT IS LEARNING TRANSFER
.........................
PROMOTING POSITIVE LEARNING TRANSFER.............................................
22
24
What Affects Learning Transfer?.........................................................
24
Effects of the InstructionalTypes on Learning Transfer...................... 25
CHAPTER 3.................................................................................................27
HYPOTHESES............................................................................................27
CHAPTER 4.................................................................................................29
EXPERIMENTAL APPROACH .............................................................
29
BRIEF DESCRIPTION OF THE EXPERIMENT.................................29
SELECTION OF THE INDIVIDUALS FOR THE STUDY..................32
DEVELOPING THE SURVEY..................................................................32
Page 4 of 4
LIST OF REQUIREMENTS FOR THE SURVEY.................................................
DESIGNING PROBLEMS FOR THE SURVEY...................................................
32
Puzzle Answers and Explanations........................................................
Section 1 ............................................................................................
33
36
36
Section 2 ............................................................................................
36
PROTOTYPE OF THE SURVEY ......................................................................
DEVELOPING THE INSTRUCTIONS ................................................................
37
37
DATA ANALYSIS........................................................................................40
TRANSFER FORMULA...................................................................................
COMPARISON OF FORMULAS ......................................................................
SELECTING THE FORMULAS FOR THE DATA ANALYSIS...............................
40
43
43
CHAPTER 5.................................................................................................45
RESULTS .....................................................................................................
45
SUMMARY OF THE SURVEY RESULTS.............................................45
EXPERIMENTAL LIMITATION ..........................................................
47
VARIABLE TEST CONDITIONS.....................................................................
SELF-SELECTION ........................................................................................
47
48
CHAPTER 6.................................................................................................50
DISCUSSION...............................................................................................50
DISCUSSION OF THE SURVEY RESULTS........................................
50
EFFECT OF THE INSTRUCTIONS ON TIME ....................................................
50
EFFECT OF INSTRUCTIONS ON THE RATE OF CORRECT ANSWERS..............51
EFFECT OF THE INSTRUCTIONS ON NUMBER OF ATEMPTS ........................ 53
EFFECT OF THE INSTRUCTIONS ON NUMBER OF GIVE UPS.........................54
EFFECTS OF INSTRUCTIONAL TYPES ON LONG-TERM MEMORY RETENTION 54
CHAPTER 7.....................................................................................................55
CONCLUSIONS..........................................................................................55
BIBLIOGRAPHY........................................................................................58
APPENDIX - COMPLETE RESULTS OF THE EXPERIMENT..........60
Page 5 of 5
LIST OF TABLES
Number
Page
TABLE 1. ANTICIPATED EFFECTS OF INSTRUCTIONAL TYPES ON KEY
LEARNING TRANSFER PARAMTERS....................................................
TABLE 2. PRIORITY OF THE REQUIREMENTS...............................................
TABLE 3. COMPARISON OF PERCENTAGE TRANSFER OBTAINED BY THREE
TRANSFER FORMULAS..........................................................................
TABLE 4. SUMMARY OF SURVEY RESULTS...................................................
TABLE 5. BREAKDOWN OF TIE ANALYZED OUTPUT FILES BY THE GROUP
TYPE .....................................................................................................
Page 6 of 6
6
28
33
44
46
48
LIST OF FIGURES
FIGURE 1. LEVELS OF INVENTION ..................................................................
9
FIGURE 2. SIMPLIFIED ARIZ DIAGRAM ......................................................
14
FIGURE 3. FLOWCHART OF THE SIT PROCESS [26] ..................................
15
FIGURE 4. TRIZ SOLUTION - GRIPPING COMPLEX PARTS WITH A VISE .... 16
FIGURE 5. EFFECT OF THE CREATIVITY BARRIER ON THE SOLUTION PATH 18
FIGURE 6. FUNCTIONAL DECOMPOSITION OF THE CREATIVITY BARRIER .. 20
FIGURE 7. DIAGRAM OF THE SURVEY PROCESS ........................................
31
FIGURE 8. FIRST PUZZLE OF THE NUMBER SERIES AS PRESENTED TO THE
CONTROL G ROUP ..................................................................................
38
FIGURE 9. FIRST PUZZLE OF THE NUMBER SERIES AS PRESENTED TO THE
GROUP 1 (GENERIC INSTRUCTIONS) ....................................................
39
FIGURE 10. FIRST PUZZLE OF THE NUMBER SERIES AS PRESENTED TO THE
GROUP 2 (SPECIFIC INSTRUCTIONS) ....................................................
39
FIGURE 11. EFFECT OF THE INSTRUCTIONS ON TIME, SECTION 1. .......... 50
FIGURE 12. EFFECT OF THE INSTRUCTIONS ON TIME. SECTION 2 ............ 50
FIGURE 13. EFFECT OF THE INSTRUCTIONS ON THE RATE OF CORRECT
A NSW ERS, SECTION 1...........................................................................
51
FIGURE 14. EFFECT OF THE INSTRUCTIONS ON THE RATE OF CORRECT
A NSW ERS, SECTION 2 ............................................................................
51
FIGURE 15. EFFECT OF THE INSTRUCTIONS ON NUMBER OF ATTEMPTS,
SECTION 1..............................................................................................
53
FIGURE 16. EFFECT OF THE INSTRUCTIONS ON NUMBER OF CORRECT
ANSW ERS, SECTION 2...........................................................................
53
FIGURE 17. EFFECT OF THE INSTRUCTIONS ON NUMBER OF GIVE UPS,
SECTION 1..............................................................................................
54
FIGURE 18. EFFECT OF THE INSTRUCTIONS ON NUMBER OF GIVE UPS,
SECTION 2..............................................................................................
54
FIGURE 19. SOLUTION PROCESS FOR SELF-DISCOVERY.............................56
FIGURE 20. SOLUTION PROCESS WITH INSTRUCTIONS .............................. 56
Page 7 of 7
Chapter 1
CREATIVITY
DeEning Creativhit
Many definitions of the word "creativity" exist and are being used.
The
Wordsmyth.net [30] dictionary gives the following definition for creativity: "the
capability of inventing or producing original or imaginative work". Roget's II [21]
definition is even briefer: "the power or ability to invent". Researchers often give
their own definition to this term. For example, D. Feldman, M. Csikszentmihalyi
and H. Gardner in their 1994 book Changingthe World: A Frameworkfor the Study of
Creafiti
[10] produce the following definition: "creativity is the achievement of
something remarkable and new, something which transforms and changes a field
of endeavor in a significant way".
Under such a definition only "high" creativity is given consideration. Only when
something of a very high caliber, like a transistor, is created, the authors argue the
creativity is exercised. I would disagree with such a narrow definition. Creativity
is much more common is our society and, furthermore, it is even required of
"ordinary" people, like students, engineers, scientists, managers, etc. on a daily
basis. For example, students display their creativity in various contests (i.e. MIT
50K
Entrepreneurship
Competition,
http://50k.mit.edu),
coursework
(i.e.
building race cars as part of the course 2.810, http://me.mit.edu/2.810),
infamous hacks (http://hacks.mit.edu/), and in many other ways. Creativity in
other areas of human activity is also quite common: creative ad campaigns are
often used in marketing to attract consumers and make the message more
Page 8 of 79
memorable. We will limit our discussion however to the realm of science and
engineering. In these disciplines, the patent database reflects the creative effort of
many thinkers.
Levels of Invention
Based on his patent research, G. Altshuller [1] differentiated between five levels
of inventions. According to his classification, Level 1 problems are the easiest
ones and Level 5 problems are the most difficult.
He gives the following
explanation: "In problems of the first level the object (device or method) does
not change (for example, the heat insulation already present is strengthened). At
the second level, the object is changed but not substantially (high reflective
surface is added to the heat shielding device). At the third level the object is
changed essentially and at the fourth level the object is changed entirely; in the
fifth level the entire technical system is changed in which the object fits."
The study of the patent database points at the following breakdown of patents by
the levels of inventions.
1%
4%
Level 1 - Apparent Solution
32%
18%
J
Level 2 -Improvement
Level 3 - Invention inside Paradigm
45%
Level 4- invention outside Paradigm
Level 5- Discovery
Figure 1. Levels of Invention
Page 9 of 79
Altshuller classifies creativeness by the type of the solution. This is a more open
and useful approach. Under Feldman et al definition of creativity only problems
of Levels 3 through 5 are being considered. This leaves out the vast majority of
problems (Levels 1 and 2 account for 77%) that most individuals encounter in
their lives.
Two Models of Innovation and Invention: Individual vs Cultural
Two different theories or explanations of the discovery process are often
discussed. One explanation is the "great inventive genius" model of discovery.
The proponents of this paradigm maintain that the inherent creative genius of
some individual's mind is responsible for the discoveries. The other theory relies
on the hypothesis that the discovery is "in the air" at that time - that the cultural
conditions are ripe for the discovery. These two explanations of discovery stem
from two opposing views of human behavior: an individualistic explanation
versus a group or social view.
Robert Haskell [15] compares the great inventive genius model to "the great
man" theory in history. According to this theory, it's the great individual who
changes history. The contrary view is that historical conditions create the "great
man". The sociocultural model of inventions includes such factors as economics,
opportunity, and support systems for promoting transfer.
There is little doubt that historical conditions are often responsible for great
innovations, but not always. One has only to examine the ideas of Leonardo da
Vinci (1452-1519), for example, his inventing the helicopter, to see that "the
times" often have little to do with creative genius. But often they probably do.
However, even when sociocultural or historical conditions are necessary, they are
not sufficient.
Page 10 of 79
R.
Haskell
quotes
Stanford
Ovshinsky,
the
inventor
of
amorphous
semiconductors: "There are a lot of people who may be smarter than I - so what
is it that makes me a successful inventor? It's got to be that I process my
information differently and draw upon my store, my environment, differently".
Let's examine the creativity process in more detail.
Creativity Process
While no one can draw an exact diagram of what is going in one's mind when
exercising creativity, the following approach may be useful in an attempt to frame
the issue. Consider an individual faced with a problem P. At some point in time,
a solution S may be developed.
P-S
This framework schematically represents that a certain stimulus P (problem)
yields a defined and expected solution S. However, it is often the case that many
individuals may come up with different solutions. This means that the "-"
represents the creative process within the individual.
The "-" may also
characterize how well the individual knows the background information, how
extensive his/her knowledge it, how widely it spreads into the other domains,
what problem solving techniques are used, etc.
"-"
Analysis of what is behind the
may also help understand why some people seem to find the solution and
others don't.
Canpeople be taught to be more creative?
Ever since serious efforts to study creativity had begun the question of increasing
one's creativity was lingering in the air. Psychologists spent a great deal of effort
studying creative people (both deceased and alive) in an attempt to deduce
Page 11 of 79
common trends in their upbringing, education, habits, likes and dislikes, etc. as if
one can imitate a creative person's diet to stimulate his/her creativity. Over the
years many approaches have been proposed ranging from recommendations on
how to be more open-minded
(Csikszentmihalyi) to detailed algorithms
advocated by the TRIZ [1, 29] and SIT practitioners [16, 23, 26].
StructuredCreativi Techniques
TRIZ Overview
TRIZ (Russian acronym for Theory of Solving Inventive Problem) is a technique
that helps approach inventive problems in a structured manner [1]. Derived on
the basis of extensive analysis of the patent database, the TRIZ methodology
encompasses a set of tools useful for engineers and others dealing with problems
of technical nature. Attempts have been made to expand TRIZ principles into
management techniques, creativity education for the children [29], etc., but they
are less successful than the core discipline.
Several notions lie at the heart of TRIZ. The principle of idealiy (defined at the
sum of all useful functions of the system divided by the sum of all harmful
functions of the system) states that all systems evolve in the direction of increased
ideality.
For example, today's automobiles have more useful functions (higher
reliability and durability, better comfort, more features, etc.) and fewer harmful
functions (cleaner emissions, lower noise, lower content of non-recyclable
materials, etc.) than the vehicles produced even 10-15 years ago. Taken to an
extreme, an ideal system from the TRIZ point of view performs the function, but
does not itself exist. This maybe difficult or impossible to achieve in real life, but
it is a good "stretch goal".
To help illustrate this point, the following example may be useful. Suppose a set
of samples of several alloys need to be tested for their resistance to a corrosive
Page 12 of 79
environment. The alloy samples are made into small cubes, deposited into vials
with acid solution and are subjected to heat and vibration to accelerate the test.
Unfortunately, the glass vials tend to crack.
samples?
What can be done to test the
Traditionally engineers will try to upgrade the vials to a higher
performing material so that they survive the test or may try to find an environemt
less aggressive to the vials. From the TRIZ point of view, however, the vial is
only needed to contain the acid solution.
It does not help to test the alloy
samples. The actual test occurs at the interface of the alloy sample and the acid
solution. Ideally the vial needs to be absent, but the acid solution needs to be
retained in some manner. How can this be accomplished? A simple way to do
this is to drill a round hole in the alloy sample and fill it with the acid solution.
Now the vial is gone and the acid solution is retained right where it needs to be.
The notion of ideality is quite general and somewhat philosophical in nature, but
it can drive the system architect to closely evaluate each component in the system,
define their useful and harmful characteristics, attempt to combine components
in order to reduce complexity, increase system reliability, etc.
Several other tools are more prescriptive in nature. For example, ARIZ (Russian
acronym for the Algorithm of Solving Inventive Problems) provides a step-by
step guide to define the problem, the ultimate desirable outcome, describe the
contradictions' that prevent one from reaching the solution and, finally, resolve
them without violating the ideality principle.
1A
contradiction is such a situation in which improving a desired parameter leads to deterioration
of some other parameter. For example, one may want to make a certain part stronger, but that
makes it heavier at the same time.
Page 13 of 79
Yes
ItNo
Figure 2. Simplified ARIZ diagram
The tools of resolving contradictions are probably the most useful. According to
the TRIZ methodology, contradictions can be resolved in one of the four
following ways: in time, in space, between the parts of the object and the object
in whole, and upon a condition. For example, resolution upon a condition would
suggest speed sensitive steering efforts in an automobile (steering effort is low
when the vehicle is moving with low speed and steering effort is higher when the
vehicle is moving faster).
StructuredInventive Thinking Overview
Developed on the basis of TRIZ methodology, Structured Inventive Thinking
(SIT), grew in its own methodology. A student of Altshuller brought the method
to Israel, where it was extensively revised and simplified, enabling the method to
be learned in a significantly shorter time, and with less reliance on external
databases.
Page 14 of 79
The SIT methodology deals with conceptual solutions to technological problems.
Its purpose is to focus the problem solver on the essence of the problem, to
enable the discovery of inventive solutions, and to make the process an efficient
one. It does this by guiding the user through either of two algorithms (see Figure
3) which structure the problem in such a way as to allow the user to bring to bear
various techniques that have been found to be helpful in inspiring creative
solutions.
SIT has been taught to over 3000 engineers in Israel, and is being used by a
number of companies there, including Motorola and Intel.
Ford is the first
company to introduce the method in the U.S. It is currently being taught at Ford
Design Institute. The courses, 24 contact hours in length, have been given so far
to over 1000 Ford engineers and scientists in the U.S. and Europe.
Collect
Information
Select
Objects
Draw
Closed-World
Diagram
Determine
Detemine
Initial & Final
States
Closed World Method
Draw
QualitativeChange Graphs
Draw
Apply
Da
pply
And/Or
Particles
Tree
Particles Method
Dimensionality
Solution
Concepts
-
Pluralization
Redistribution
Figure 3. Flowchart of the SIT Process [26]
Page 15 of 79
Unique"
ness
Summary of Structured Creativit Techniques
Structured creativity techniques, are useful tools to help engineers and scientists
develop creative solutions. Commercially available software products simplify
database searches and provide pictorial examples of creative solutions to
problems similar in nature. For example, if someone is trying to solve the
problem of gripping parts of complex shape with a vise, the software tool called
Innovation WorkbenchTM distributed by Ideation International Inc, will suggest
the adding intermediary elements that can conform to the complex shapes, yet
effectively transfer the gripping force as illustrated in Figure 4.
Figure 4. TRIZ Solution - Gripping Complex Parts with a Vise
A product utilizing a similar principle appeared recently on the market. The
Gator-Grip@ socket (http://www.gator-grip.com) claims to grip "anything that
isn't round"!
Typically the user will benefit greatly from attending a course or workshop where
the basic principles of these methodologies are reviewed.
Page 16 of 79
Creativity Barrier
Problem solving is sometimes hindered by the Creatioiy Barrier, which prevents
the individuals from choosing the direct path to the solution.
Figure 5A
illustrates the problem solving process influenced by such a creativity barrier: the
efforts of the individual to solve this problem are halted by the creativity barrier.
Depending on the difficulty of the problem, this can be a more or less permanent
position. Research shows that the more difficult the problem, the more attempts
are required in order to solve it. Often after many trials and failures, a solution
path is finally found. This is shown in Figure 5B where the solution path goes
around the creativity barrier. This process is characterized by extended time and
fruitless trials. If the individual is successful in breaking the creativity barrier,
he/she is able to attain the solution much more directly and faster (Figure 5C).
The difference between the approach in Figure 5B and Figure 5C is in the
fundamental level of understanding the challenge and in the ability to face the
root cause. The approach depicted in Figure 5B is usually referred to as trail-anderror. Typically multiple solution attempts will be emanating in various directions
from the node P and one of them may eventually yield solution S. Figure 5C
shows the process of someone who can pinpoint the root cause of the problem
and attack it directly.
We will attempt to measure this process. In the experiment described in more
detail in Chapter 4, we will ask a group of individuals to solve a series of puzzles,
while taking measurements of time, the number of attempts, the rate of give ups,
and, of course, the success rate.
Page 17 of 79
Solution
Creafivi_*
|e
.
Creativiy
4
Solution0
Q~r
0
Barrier
Solution
Solution
Solution
u
-.
Barrier
path
t
4
0
attempt
Problem
Problem
Problem
C) Break through
B) Solution path found around
A) Creativity process
the Creativio Barrier
the CreativioBarrier
hindered by the Creativity
Barrier
Figure 5. Effect of the Creativity Barrier on the Solution Path
Page 18 of 79
The notion of the Creativity Barrier can be illustrated with the help of this
familiar example. The task is to connect all the nine dots with four straight lines,
without lifting the pen from the paper.
*
0
0
*
0
0
Zander [31] describes the experience of someone solving this puzzle for the first
time: "...you will most likely find yourself struggling to solve the puzzle inside the
space of the dots, as though the outer dots constituted the outer limit of the
puzzle." We look at the dots and all we can see is a square. We then make a
typical mistake.
This situation is similar to the one shown graphically in Figure 1A. Of course,
this is not the right solution. What's needed to solve this puzzle is to abstract
from the outer dots and expand the solution space. We need to move ourselves
from the hopeless situation in Figure 1A to a desirable situation in Figure 1C. As
soon as one realizes that the instructions did not contain anything about fitting
the lines nithin the area staked out by the outer dots, and the entire white sheet
can be used, the creativity barrier begins to crumble. The reader is encouraged to
attempt to solve this puzzle before proceeding to the next page, where one of the
possible solutions is shown.
Page 19 of 79
It is thought that creative people are less affected by the creativity barrier and,
therefore, are capable of arriving at the solutions faster and more reliably than
others. Of course, the other way to look at the so-called "creative" people is to
say that they are more capable of expanding their solution space. So, it follows
that the lower one's creativity barrier or the more one is capable of consciously
destroying it, the more creative the person is. But what affects one's creativity
barrier and how does one go about lowering it?
The creativity barrier can be viewed as composed of two primary ingredients:
personalinhibiionsand the context.
Creativity
Barrier
Individual
Inhibitions
Context
Groove-in
Setting
Practice
Pressure
Type of transfer
Expectations
Fear of failure
Risk aversion
Other
Other
Figure 6. Functional Decomposition of the Creativity Barrier
Page 20 of 79
To enhance the understanding of the notion of the creativity barrier it is
important to figure out how the two building blocks interact, what is the balance
between them, does this balance have a dynamic nature, what may be the
circumstances that cause this balance to shift in one direction or another.
The individual inhibition is a function of one's prior experience (groove-in),
practice with similar type of problems, type of knowledge transfer, fear of failure
or its consequences, etc. The context has to do with the environment, the setting
and pressure that may come from the desire to fulfill the expectations of others,
fear of saying or doing something that may cause others to not accept it or, even
worse, judge or make fun of you, etc.
In this thesis I will focus on the individual inhibitors and, in particular, on what
can be done to improve the knowledge transfer. I will use the terms "knowledge
transfer" and "learning transfer" interchangeably.
How to reduce the creativity barrier
While many ways to address each and every one of the elements shown in Figure
6 may be devised, this thesis centers on the investigation on how the creativity
barrier can be lower by influencing just a single factor - the transfer of learning.
From that point of view, I will examine how the three primary mechanisms of
learning transfer, namely self-discovery, instruction and tutoring, affect creativity
and problem solving skills.
Page 21 of 79
Chapter 2
TRANSFER OF LEARNING
Processes of learning and the transfer of learning are central to understanding of
the development of important competencies.
Since early childhood people are
exposed to various types of learning experiences: instruction, tutoring, selfdiscovery, etc.
Knowledge and skills acquired through these various types of
experiences leads to varying levels of proficiency.
What is Learning Transfer?
Transfer of learning means that experience or performance on one task
influences performance on some subsequent task. Transfer of learning may take
three different forms: (1) performance on one task may aid or facilitate
performance on a second task, which represents positive transfer, (2) performance
on one task may inhibit or disrupt performance on a second task, which
represents negative transffer, and (3) finally, there may be no effect of one task on
another, in which case we have an instance of Zero transffer [9]. His study showed
that "students who have thoroughly mastered the principles of algebra find it
easier to grasp advanced work in mathematics such as calculus." Another study
[2], compared students learning LISP as a first programming language to students
learning LISP after having learned Pascal. The Pascal students learned LISP much
more effectively, in part because the appreciated the semantics of various
programming concepts.
For effective positive transfer to take place [19]:
Page 22 of 79
1. The student must understand that the learned behavior can be generalized to
other domains
2. It is necessary for the student to mindfully abstract or decontextualize the
schema from the learned behavior so that it can be modified and applied
3.
The student needs to recognize the relevant sameness between the
instructional situation and a transfer situation.
Ability for abstract thinking as an important ingredient for problem solving and
creativity.
Negative transfer may also take place.
In the case of the negative learning
transfer the previously acquired skill will prevent the individual from performing
well on the new task. This may be a result of overleaming leading to lack of
flexibility in thinking. Extensive experience in a certain field may give rise to such
a dichotomy. For example, those visiting the U.K. for the first time often have
difficulty navigating through traffic. The power of habit of first looking to the
left and then to the right when crossing the street does not work well when the
traffic moves in the opposite direction than one is used to coming from the U.S.
or continental Europe. In psychology this is referred to as automaticity.
Even though in problem solving we are dealing with a higher order cognitive
functions, the basic principle still applies. On the one hand deep expertise may be
required to perform the task well, but on the other hand, the same experience
may tend to lock the individual in a particular frame of mind, thus contributing to
negative transfer. To counter this, the "fresh eyes look" approach is often called
into action, which entails bringing a less experienced person to analyze the same
problem. In this case the less experienced person, who is not as constrained by
conventional wisdom, may offer new perspectives and help the situation.
Page 23 of 79
Promoting Positive Learning Transfer
What Affects Learning Transfer?
Leaming transfer has been studied extensively since early 1900's. Here I present
a very brief summary of the key points. Much of this is based on [4].
Several critical features of learning affect people's abilities to transfer what they
have learned. The amount and kind of initial learning is a key determinant of the
development of expertise and the ability to transfer knowledge.
While time on task is necessary for learning, it is not sufficient for effective
learning. Time spent learning for understanding has different consequences for
transfer than time spent simply memorizing facts or procedures from textbooks
or lectures.
The context in which one learns is also important for promoting transfer.
Knowledge that is taught in only a single context is less likely to support flexible
transfer than knowledge that is taught in multiple contexts. With multiple
contexts, students are more likely to abstract the relevant features of concepts
and develop a more flexible representation of knowledge. The use of well-chosen
contrasting cases can help students learn the conditions under which new
knowledge is applicable. Abstract representations of problems can also facilitate
transfer. Transfer between tasks is related to the degree to which they share
common elements, although the concept of elements must be defined
cognitively.
All new learning involves transfer. Previous knowledge can help or hinder the
understanding of new information. For example, knowledge of everyday
counting-based arithmetic can make it difficult to deal with rational numbers;
assumptions based on everyday physical experiences (e.g., walking upright on a
Page 24 of 79
seemingly flat earth) can make it difficult for learners to understand concepts in
astronomy and physics and so forth.
Effects of the InstructionalTypes on Learning Transfer
In this thesis we will demonstrate that the knowledge transfer is also affected by
the instructional type. Specifically, three various approaches will be considered.
1.
Generic instructions.
If the individuals are instructed to apply certain
knowledge in a hypothetical situation, it is hopeful that when they encounter the
situation similar to the "designated" one, they will apply the knowledge and
achieve a successful result.
To achieve the successful outcome however, the
individuals must: 1) recognize that the situation is of the type when this particular
knowledge must be applied; 2) invoke the particular instructions in their mind
that relate to this situation; and 3) apply knowledge in the correct way.
The
likelihood of the success depends on how well the instructions were received,
how explicit they were, how extensive the knowledge is (this is particularly
important if the encountered situation is somewhat different from the 'textbook'
version and a certain amount of knowledge manipulation is required) and how
proficient the individual is with the actual knowledge application. It is possible
the individuals will generate creative solutions in this situation, however,
following specific instructions is likely to yield a predictable result.
2. Specific Instructions (Tool). Another method of invoking knowledge
transfer is through the use of a specialized tool. Such a tool could be in the form
of detailed, step-by-step instructions or a software product, cue cards, etc.
Evidence suggests that a tool can be highly effective in the hands of a well trained
individual and will allow him or her to produce a large number of solutions in a
quick manner. The tool is much less effective for individuals lacking training. In
both instances, however, over reliance on the tool is possible. Another drawback
Page 25 of 79
of using a tool is for non-standard type situations when the effectiveness of the
tool is substantially diminished.
3.
Self-discovery will require the most creativity from the individual and it is,
probably, the least certain method. Success in self-discovery stems from the most
in depth understanding of the subject matter, an insight and/or discovery of an
underlying
trend.
This
in depth
understanding
is
achieved
through
experimentation with a wide range of solution directions and a deeper dive into
them.
If the individual is successful in achieving the solution, it is likely to
remain in memory the longest. Even if the individual forgets the solution after a
period of time, he/she is likely to develop this solution once again if required as
long as the knowledge of the subject matter remains active.
Page 26 of 79
Chapter 3
HYPOTHESES
1.
It is hypothesized that the learning transfer is affected by the method by which
the individuals acquire the skills needed to solve the problem.
Three
distinctive methods are identified and compared in this study: 1) selfdiscovery, 2) generic or process level instructions and 3) a tool or very
narrow and specific level instructions.
2. It is hypothesized that the quality of learning transfer can be measured by
the speed and the correctness of the responses.
a.
The individuals using the self-discovery approach will require more
time initially, but as they acquire the fundamental understanding of
the subject matter through a more thorough investigation of a wider
range of appraoches, will take progressively less time.
When
presented with a problem of a slightly different nature, but utilizing
the same underlying principle, they will recognize the fundamental
similarity and will be well poised to apply their knowledge to solve
this problem.
These individuals will solve the non-standard
problem faster and with a higher percentage of correct answers than
those using methods 2 and 3, described above.
b. The individuals using generic instructions will take less time initially
than those using the self-discovery approach as the fundamental
principle is already extracted for them.
If the application of this
distilled and readily available fundamental principle is clear to them,
they solve the initial problem faster than those practicing the selfdiscovery approach. They are also likely to get a higher percentage
of the correct answers on the first attempt. When presented with a
Page 27 of 79
problem of a different nature, but utilizing the same fundamental
principle, these individuals will be less likely to apply their
knowledge than those using the self-discovery approach since the
creative step needed for this exercise was not practiced by them with
the previous problems.
c. The individuals using the tool, or specific instructions, will perform
well when the application of the tool is transparent.
They will
exhibit the fastest time on the first problem and the highest
percentage of the correct answers on the first problem. However,
their performance on the problem of a different nature, but utilizing
the same fundamental principle, will be markedly worse than of
those practicing the self-discovery or those receiving the specific
instructions. The creativity of the individuals using the tool will be
hindered by the excessive reliance on the tool.
3.
It is hypothesized that the method of skill acquisition also affects the longterm memory retention. Those practicing the self-discovery will have better
long-term memory retention than those receiving specific instructions, with
those receiving generic instructions falling between the other two categories.
However, this aspect of learning transfer is not a subject of this thesis.
The following table will help summarize the hypotheses described above.
SelfDiscovery
Generic
Instructions
Specific
Instructions
High
Medium
Low
Thought flexibility
High
Medium
Low
Speed
Low
Medium
High
Learning Transfer
Parameter
Level of
understanding of
the subject Matter
Table 1. Anticipated Effects of Instructional Types on Key
Learning Transfer Paramters
Page 28 of 79
Chapter 4
EXPERIMENTAL APPROACH
BriefDesciption of the Experiment
The experiment was devised to quantify the effect of the method by which the
learning skills
are acquired.
This
experiment involved
three groups of
respondents:
1)
Control group. The individuals in this group received no instructions on
how to solve the problems. These individuals were forced to use the selfdiscovery approach, although it was not communicated to them.
2) Test Group 1. The individuals in this group received generic instructions
on how to solve the first problem in each of the series.
3)
Test Group 2. The individuals in this group received spedfic instructions
on how to solve the first problem in each of the series.
It is important to note that in the Groups 2 and 3 the respondents received
instructions for only the first problem in each of the two series.
The performance of each of the respondents was measured using several key
parameters for each of the puzzles (a total of 9 puzzles arranged in two series
were presented to each of the respondent):
1)
Time. The time to solve the puzzle was measured and recorded to the
output file based on the computer internal clock.
Page 29 of 79
2)
Success rate on the first attempt.
If the respondent was able to
develop the correct answer on the first attempt, the value of 1 was
recorded to the output file. If the respondent entered a wrong answer
on the first attempt the value of 0 was recorded to the output file.
3)
Ultimate success rate. If the respondent was able to develop and
enter the correct answer the value of 1 was recorded to the output file.
If the respondent was not able to develop the correct answer and gave
up the value of 0 was recorded to the output file.
4) Number of attempts undertaken in a quest to develop the final correct
answer.
5) Give up rate. If the respondent opted out of solving the problem and
hit the "Give Up" button, the value of 1 was recorded to the output file.
The individuals were contacted via e-mail. The e-mail contained a request to
download the attached file, run the program and e-mail the results back for
compilation of the data and analysis. The flowchart of the process is shown on
page 31.
The individuals had several opportunities to opt out of the survey. For example,
they may have disregarded the initial e-mail all together. A variety of reasons may
have led the person to this decision: too busy, not interested in helping out, etc.
The next opportunity to drop out was after the start of the program. When the
respondents got the first glimpse of the puzzles, they made a decision on whether
to proceed or quit. Some people found the puzzles of mathematical nature of
little interest or they may have disliked them based on the prior experience. Yet
another opportunity to opt out was any time throughout the survey process. The
respondents may have thought that the problems were too difficult, or they have
Page 30 of 79
already spent enough time, or it simply required more time commitment from
them than they originally anticipated. The last decision point on whether to go
through with the survey or to opt out arose upon completion of the survey. The
respondents had an opportunity to view their output file and make a decision on
whether to send this file for analysis or not.
Individual is
contacted by
e-mail
---- ------------ +
Opt out
Individual runs
the program
Random
assignment
-----------------+
Control group
(No Instructions)
Opt out
Group 1
Group 2
Instructions)
Instructions)
(Generic
(Specific
opt out
O---------------+
Individual
solve the
puzzles
-------------------+
Opt out
Individual emails results
file for analysis
Figure 7. Diagram of the Survey Process
Page 31 of 79
Selection of the Individuals for the Study
Since the goal of the study was to teach the participants a certain skill using three
distinctive methods and then to gauge how effectively they learned this skill, it
was important to select the individuals open to learning. At the same time, it was
important to select a relatively homogeneous group of people, so that no
significant advantage can be gained from having prior knowledge and or skill.
Based on these considerations, it was decided that graduate students at MIT
Sloan and Engineering Schools, Haas Business School at University of California
at Berkeley, and engineering professional at Ford Motor Company and a several
other organizations, would be targeted. It was decided that approximately 100
output files need to be collected and analyzed to ensure statistical power of the
data.
Developing the Survey
Applying the principles learned in System and Project Management as well as in
Systems Engineering, the first item of priority was to define the requirements for
the survey.
List of Requirements for the Survey
The following set of the requirements was identified and prioritized based on the
available resources, timing and expected level of commitment on the part of the
respondents.
Page 32 of 79
Assessment of learning transfer
Gauge the effectiveness of learning transfer
High
Teach a skill in the course of the survey
High
Measure the effectiveness of the learning
transfer of a somewhat different task
High
Repeat previous two steps for another set
of problems
High
Measure respondent on at least two scales
High
Time
High
Correctness of the response
High
Ease of use
Provide fun and excitement for the respondent
Takes no more than 10 minutes to complete
Save data for analysis
Run on PC, Mac or Unix platform
High
Medium
High
High
Provide information about the respondents
Demographics
Low
Education level
Low
Educational background
Low
Table 2. Priority of the Requirements
Designing Problems for the Survey
Selecting the problems for the study was the crucial task. The problems have to
have a certain amount of commonality between them so that the respondents
could practice with them while acquiring the skill, and, at the same time one of
the problems needs to be of a similar type, yet different enough to allow the
respondents a chance to transfer the learning. So, the series of the such problems
was represented as follows: A, A', A", A', B. In this series the problems A, A',
A", and A"' share common features, while the problem B although based upon
the same underlying principle, is substantially different.
Page 33 of 79
A number of various problems were considered. In the end, it was decided that a
the first set of problems will comprise of a number series and the second set of
problems will be more graphical and involve a series of triangles with numbers
forming a certain pattern arranged at the peaks of the triangle and in the center.
In the number series of the puzzles the respondents were asked to determine the
next number in each of the strings. The following puzzles were used (Part 1 of
the survey):
A) What is the next number in this series?
2, 5, 14, 41
B) What is the next number in this series?
84, 80, 72, 60
C) What is the next number in this series?
39, 50, 63, 78
D) What is the next number in this series?
55, 74, 57, 72, 59
E) What is the next number in this series?
144, 12, 120, 10
Puzzles were presented one by one to the respondents so that they couldn't easily
cross-reference them. The respondents were informed whether the entered
solution was either correct or wrong; they were not allowed to go back to a
particular puzzle once they either entered the correct answer or gave up.
Page 34 of 79
In the triangle series of the puzzles the respondents were asked to determine the
value in the center of the last triangle in each of the strings (Part 2 of the survey):
A)
3
7
4
A2
2
A
2
3
5
9
A2
2
B)
2
4
6
A
3
2
A
1
3
5
3
C)
3
2
D)
4
4
3
2
G
D
E
2
5A
7
D
AF
F
A
C
Page 35 of 79
E
G
PuZZle Answers and Explanations
Section 1
A) The difference between the numbers in this series represents a power series of
3 (5-2=3; 14-5=9, 41-14=27 or 31, 32, 3). So, the difference between 41 and the
last number in the series should be 34=81, making the last number 41+81= 122
B) The difference between the numbers in this series is: 4, 8, 12. Clearly, this a
arithmetic progression, increasing by 4. So, the next delta should equal 16. This
makes the last number in the series: 60-16=44.
C) Similarly to B, the delta between the numbers in the series is 11, 13, 15. The
next odd number is 17, making the last number in the series 78+17=95.
D) There are two series embedded into this string of numbers. One series is 55,
57, 59 which is increasing by 2. The other series is: 74, 72, ?, which is decreasing
by 2. So, the last number in the series is 72-2=70.
E) This series can be solved in the following manner: 144 divided by 12 (a
constant) is 12, which is the next number in the series after 144. If the result of
the division operation is then multiplied by 10, it yields 120, which is the next
number in the series. Similarly, 120 divided by 12 (same constant) yields 10 - next
number in the series. 10 multiplied by 10 (equals 100) produces the answer to the
puzzle.
Section 2
A) Adding the numbers at the comers of the triangle yields the solution: 14.
B) Multiplying the numbers at the corners of the triangle yields the solution: 15.
Page 36 of 79
C) Multiplying the number at the top of the triangle by the number at the bottom
right hand comer and subtracting the number at the lower left comer yields the
solution: 12.
D) Converting the letters into numbers and manipulating the numbers as
described in C), yields the answer: W.
Prototype of the Survey
Two main platforms for conducting the survey were considered: the web-based
and a stand alone program. Each has its own advantages and disadvantages. For
example, the web-based survey is easy to create, easy access and it allows
automatic data compilation. The main challenge with the web-based approach,
however, is that the variations in network traffic density can substantially affect
the time calculation. Since the time is one of the main measures of the learning
transfer, it was decided to use the stand alone program to ensure the high quality
of the time data, even though this approach does not allow for as easy of an
access or automatic data compilation.
The web-based prototype was used early in the development (beta testing 1). The
goal was to ensure that the respondents can solve the puzzles in the reasonable
amount of time and that the instructions were clear.
Developing the Instructions
The importance of this step should not be underestimated. The instructions for
Group 2 should be such that they convey the general principle useful to solve any
of the problems in the given series. On the other hand they can't be specific too
specific because then the difference between the Groups 2 and 3 will disappear.
Page 37 of 79
The following screen captures illustrate the varying levels of instructions used for
the Control Group and Groups 2 and 3.
Conskier the sequence of numbers below-
By conducting mathematical manipulations (addition, subtractin,
multiplication, etc.) deduce the formula that links the numbers. Apply
this formula to determine the next number in each of the series
Please type In the number (and press the return key)
Figure 8. First puzzle of the number series as presented
to the Control Group
Page 38 of 79
Think at the misakig number in terms of a trend of numbers. Is the
trend bntre n or de"'Ing? Can you determlne anotlw p*tnm?
If t etnd is kh",
how apidty do the numbws incen? Now
think of the mattwinaical functions tht can egain such a behavior.
For example, i the trendnates a rapi 1inCRfse, It could te
explned by muApltatot, pOer lvw, etc, while a slower nscxdIng
trend could be explained by sumrntict Dedue thefvmula that
IkfS the numbers and determine the net number r the series.
For exampte, coside the nqwnce of numnt beow
2, 5, 14, 41,7
This s a rapidy ascending trend; the diference between the
numbers repst* a P4*we srist Apply tis ormula to d
the net number wi each o# the senws
m*W
e
Please type in the number (anid pre the return key)
Figure 9. First puzzle of the number series as presented
to the Group 1 (Generic Instructions)
Consider the sequence of nurabets WeOW By COMdWC~ng
tna#t4&mAIc manopulatian" (addifion, wsbion, mutlilcatlwn, etcj
deduce ttw- formula that links the numbers. Appty this tforula to
determine te nrxA numbef In each of the wsft,
The dstmnoe bybwlen the numbeTs iM this series repsents a powe
series of 3 (5-2=3=; 14-5=9=32 41-14=27=3J So the dfrence
between the last number ki the senes and 41 should equal 34 (3 to
Ine pownr 4) 0r81 lTau the answr to the put.l is z-41=41 or
=122
5, 14, 41,
Piveae type thea number t(nd press the return ky)
Figure 10. First puzzle of the number series as presented
to the Group 2 (Specific Instructions)
Page 39 of 79
Data Analysis
Approximately 250 individuals were contacted by e-mail and 90 output files were
collected.
Transfer Formula
The amount and direction (positive or negative) of transfer is determined by
employing one of several formulas. The three transfer formulas described below
are similar in that they involve making comparisons between the experimental
and control groups on performance on the transfer task.
In order to apply a transfer formula to a given set of data, some measure of
performance must have been taken. Measures frequently used include: (1) the
number of trials required to reach a given level of mastery; (2) the amount of time
required to reach a given level of mastery; (3) the level of mastery reached after a
given mount of time or number of trials, such as the number of correct
responses; and (4) the number of errors made in reaching a given criterion of
mastery.
A simple transfer formula is described below. Let E represent the mean
performance of the experimental group on the transfer task (Task B) and let C
represent the mean performance of the control group on the transfer task (Task
B).
By comparing the difference between E and C groups with C itself a
percentage transfer formula can be expressed as follows:
Percentage of Transfer =
C
* 100
(1a)
This formula is appropriate if the measure of performance is such that the larger
the value of the measure, the better the performance. For example, if the measure
of performance is the number of correct responses, then the formula is
Page 40 of 79
appropriate because the number of correct responses becomes larger with better
performance.
Formula (1a) will be illustrated with a simple example. Suppose we conduct a
transfer experiment in which we measure the effect of taking French this year on
the taking of German next year. In other words, we want to know if taking
French will aid or interfere in the subsequent learning of German. We employ
two groups: an experimental group that studies French for a year and then takes
German the following year and a control group that studies only German. In this
instance, Design I is employed. A measure of performance is taken on the first
test on German and we discover that the E group averages ninety correct
responses whereas the C group averages only seventy-five correct responses on
the test. Applying Formula (1a) and substituting the values for E and C, we
obtain:
9075* 100 =
75
* 100 = 20percent transfer
75
The E group shows 20 per cent transfer, which means that the E group performs
20 per cent better in German compared with the C group. Of course, we do not
know if the positive transfer is a result of the specific features of French or of
learning to learn; it is likely a mixture of both.
Formula (1a) must be modified by reversing the numerator to C - E if the
measure of performance is such that the smaller the value of the measure, the
better the performance. In this case, the formula becomes:
Percentage of transfer = CE*
C
10 0
Page 41 of 79
(1b)
This formula is appropriate with such measures as errors, trials to reach some
criterion, or time. It is obvious that as errors, trials, or time are reduced in value,
performance improves.
A second type of transfer formula was proposed by Gagne et al. (1948). This
procedure compares the difference between the E and C groups with the
maximum amount of improvement possible on the transfer task. The maximum
improvement possible is indicated by the difference between the total possible
score on Task B and the performance of the C group on Task B. If the measure
of learning is one such as number of correct responses, as in Formula (la) , and T
stands for the total possible score, the formula is
Percentage of transfer = E-C * 100
T-C
(2a)
The denominator and numerator are reversed if the measure of learning is one
such as time, trials or errors, as in Formula (1b).
Percentage of transfer = CE* 100
C-T
(2b)
A chief difficulty with using either Formula (2a) or Formula (2b) is that we do not
always know the total possible score T, and its determination may be difficult or
impossible.
Murdock (1957) has suggested a third type of transfer formula which has a
distinct advantage over the first two described.
The maximum amount of
positive transfer which can be obtained is 100 per cent transfer and the maximum
amount of negative transfer is -100 per cent; in other words, the upper and lower
limits are equal, and positive and negative transfer are symmetrical. This is
Page 42 of 79
accomplished by making the denominator of the formula include
the
performance of the E group as well as the G group. The formula is:
Percentage of transfer = F-C * 100
E+C
(3a)
Like Formula (1a), Formula (3a) is appropriate if the measure of performance is
such that the larger the value of the measure, the better the performance. If the
measure of performance is such that the smaller the value of the measure, the
better the performance, the formula must be modified to read:
Percentage of transfer= CF *10 0
E+C
(3b)
Comparison of Formulas
A comparison of Formulas (la), (2a), and (3a) is shown in Table 3, p. 44.
Hypothetical values for E, G, and T are listed along with the percentage transfer
obtained with each formula. Because different percentages of transfer are
obtained with each formula, the importance of knowing what transfer formula
was used in a particular study becomes obvious, especially if one wishes to
compare the magnitude and direction of transfer obtained in different studies.
This latter point has been strongly emphasized by both Gagne et, al. (1948) and
Murdock (1957).
Selecting the Formulas for the Data Analysis
Since the total possible score (T) is unknown in the types of problems used for
the study in this thesis, the application of formulas 2(a) and 2(b) is not possible.
Also, since we are interested in determining the relative performance of the three
groups (Control Group and Groups 1 and 2), and not in establishing the upper
Page 43 of 79
and lower control limits, the choice of formula becomes quite obvious.
The
Formulas 1(a) and 1(b) will help us quantify the effect of learning transfer.
Table 3. Comparison of Percentage Transfer
Obtained by Three Transfer Formulas
Number of Correct Responses
Percentage Transfer from Formula
E
C
T
(1a)
(24)
(3a)
50
0
50
+Infinity
+100
+100
25
15
50
+67
+29
+25
15
15
50
0
0
0
15
25
50
-40
-40
-25
0
50
50
-100
-Infinity
-100
Page 44 of 79
Chapter 5
RESULTS
Summary of the Survey Results
The complete set of survey results can be found in Appendix.
The survey output files were received from 90 respondents.
Individual output
files were examined and the outliers excluded from the analysis (see Experimental
Limitation section, p. 47). After the outliers were excluded, 84 "good" output
files were analyzed. The tables below summarize the learning transfer for the first
and the last puzzles in Sections 1 and 2. The learning transfer values for time,
number of attempts and give ups were calculated according to the formulas 1(b)
since the lower value points at a better outcome.
The values for number of
correct answers on the first trial and the ultimate number of correct responses
were calculated using formula 1(a).
The following abbreviations are used in the tables below:
C - Control Group using self-discovery
El (G) - Experimental Group 1, using Generic instruction
E2 (S) - Experimental Group 2, using Specific instruction
Transfer 1 - Learning transfer for El
Transfer 2 - Learning transfer for E2.
Page 45 of 79
Section 1, Question 1
Transfer 1 Transfer 2
C
El (G)
E2 (S)
Time, sec
93.000
121.000
91.000
-30%
2%
Correct 1
0.625
0.667
0.778
-24%
Correct
0.818
0.909
0.926
-27%
-5%
2%
# attempts
1.152
1.333
1.185
-16%
-3%
Give ups
0.030
0.042
0.000
-40%
100%
Section 1, Question 5
Transfer 1 Transfer 2
C
El (G)
E2 (S)
Time, sec
98.456
59.653
47.043
39%
52%
Correct 1
Correct
0.545
0.848
0.333
0.625
0.407
0.667
-39%
-26%
# attempts
Give ups
1.545
0.121
1.958
2.000
-27%
-25%
-21%
-29%
0.250
0.296
-107%
-145%
Section 2, Question 1
Time, sec
Correct 1
C
24.157
0.970
El (G)
65.007
0.958
E2 (S)
30.227
0.963
Correct
# attempts
Give ups
1.000
1.061
0.042
0.958
1.000
0.000
0.963
1.000
Transfer 1 Transfer 2
-25%
-169%
-1%
-1%
-4%
-4%
6%
6%
0.000
100%
100%
Section 2, Question 4
Transfer 1 Transfer 2
C
El (G)
E2 (S)
Time, sec
Correct 1
Correct
# attempts
95.673
0.273
0.515
1.788
121.099
0.250
0.417
3.542
123.115
0.296
0.667
2.519
-27%
-8%
-19%
-98%
-29%
8%
30%
-41%
Give ups
0.364
0.458
0.148
-26%
59%
Table 4. Summary of Survey Results
Page 46 of 79
The values in the columns C, El and E2 represent the mean values based on the
analysis of output files falling into the respective categories.
Learning transfer for experimental Group 1 (Transfer 1) - those using Generic
instructions - is mostly negative with the single exception of time (speed) for the
last puzzle in Section 1 (negative transfer for number of attempts and give ups
indicates more attempts and give ups respectively).
Learning transfer for the experimental Group 2 (Transfer 2) - those using specific
instructions - appears to be more ambiguous. The instructions helped the Group
2 solve the puzzles faster than the control group in the first section, but served as
a detriment in other measured attributes (time, number of correct responses on
the first trail, ultimate number of correct responses, number of attempts and
number of give ups). The situation changed for the second section where the
instructions adversely affected the speed and number of attempts, but improved
the rate of correct responses and allowed for fewer give ups.
Expermental Limitation
Variable Test Conditions
The nature of the experiment required that the respondents take the survey at in
the environment of their choice.
Varying ambient noise level, and other
conditions may have affected the level of focus on the part of the respondents.
Among the outliers were those output files in which the time for a particular
question was substantially longer than anticipated or than it took this respondent
to answer a similar question in the survey. For example, in one of the files it took
the responded nearly 27 minutes to answer question number 3 in part 1. Such an
extended time may be explained by a distraction on the part of the respondent.
In fact, one of the respondents mentioned to me that as he was taking the survey
Page 47 of 79
in his office, he received a visitor who engaged him in a conversation, thus
distracting the respondent from the survey. This particular respondent did not
read the instruction carefully enough to realize that he was being timed.
Self-Selection
As indicated in Figure 7, the respondents had several opportunities in the course
of the survey to opt out. Every decision point contributed to the self-selection.
The last decision point was probably the most critical. Looking at their results,
the individuals assessed their own performance on the survey. Their selfassessment at this point was very subjective as they had no reference point and
didn't know the average results or results of others who tool the survey.
Nevertheless some respondents may have decided that their results are not
adequate and may have elected not to send them in for analysis. The fact the
author of the thesis personally knew most of the respondents exerted further
pressure on them. Those who decided to refrain from returning the surveys, may
have felt embarrassed about their performance and preferred not to reveal it so
that the author of this thesis does not think negatively of them. This type of selfselection may have affected the data set, decreasing the population of poor
performers. It is hard to tell now which one of the three groups had the largest
number of the dropouts.
The table below provides the breakdown of the 84 analyzed output files by the
Group type.
Group
Control Group
Experimental
Group 1
Experimental
Group 2
Count
33
24
27
Table 5. Breakdown of the Analyzed Output Files by the Group Type
Page 48 of 79
Curiously enough, the most surveys returned came from the Control Group. The
obvious question arises: is such an outcome a result of the random nature of the
process of assigning the respondents to one of the three groups or is this
distribution affected by the self-selection? In other words, assuming that equal
number of people got assigned to each of the three groups, it is possible that the
individuals in the Control Group are more likely to return their output files than
those in the experimental groups? To answer this question from the statistical
point of view, one needs to compare the actual count numbers to the expected
value. If a total of 84 surveys were assigned randomly to three groups, one would
expect to see 28 surveys in each of the groups. However, in our case, we end up
with a distribution of 33, 24 and 27. So, what is the probability of getting 33
when 28 is expected? The analysis shows that such a probability is approximately
10%. Similarly, the probability of getting 24 when expecting 28, is approximately
15% (probability of getting 28 is 50%).
Although the probability of such a distribution is still within the random nature
of the process, the slight shift in favor of the control group is obvious. This leads
to the conclusion that the individuals in the control group feel somewhat better
about their result and are more likely to return their output files. One possible
explanation to this is that the expectations of solving the puzzles among the
respondents in this group are closer to the reality than for those in the
experimental groups, even though the latter ones were not told that they belong
to the experimental groups. The only way to get around this issue is to conduct
the experiment in the controlled environment, were all the respondents must
return their output files.
Page 49 of 79
Chapter 6
DISCUSSION
Discussionof the Survey Results
The survey results once again demonstrate that the problem solving activity is a
multidimensional process.
Effect of the Instructions on Time
The two plots below show time normalized with respect to the control group.
3.0-
3.0
Group 2 (Specific Instructions)
. 1.5
-
- - - - - - -
2.0
.
1.5 - - - -
-
i
- - - -
-
- - -
-
-*-
- --
- -- - -
2.5-
-
2.5 - - - - - - - - - - - - - - - - - -4-Control Group
j: 2.0 -Group 1 (Generic Instructions)
- -
- -
0.5
-
-
0.5
0.0
0.01
Puzzle 1
Puzzle 1
Puzzle 5
Figure 11. Effect of the
Instructions on Time, Section 1.
-4- Control Group
-U-w-Group 1 (Generic Instructions)
--Group 2 (Specific Instructions)
-
1.0 ----
Puzzle 4
Figure 12. Effect of the
Instructions on Time. Section 2
It can be seen that respondents in the Experimental Groups spend more time
thinking about the puzzles initially. However, when they get to the last puzzle in
the series, they spend much less time than before.
In fact, in Section 1 they
spend less time than those on the Control Group and the Section 2, even though
Page 50 of 79
they spend more time than the Control Group, they have cut down very
significantly on their time allocation.
NOTE: The time data may be confounded by the fact that the respondent in the
Experimental Groups had to spend the time on reading the instructions, while
the respondents in the Control Group did not have to do that. Since it is unclear
how much time on average the respondents in the Experimental Groups spent
reading and thinking over the instructions, the actual solution time cannot be
extracted from this data set. To get around this, a more detailed analysis of time
spent on each of the problems in series may be required.
Effect of Instructions on the Rate of Correct Answers
1.4
1
0
1.2
2
S 1.0-
0
00
- - -
-
-
-
0
1.4
1.2 - - - - - - - - - - - - --
s0.8---S4
0.2
Z
0.0
- - - - - - - - -4- Control Group
- Group 1 (Generic Instructions)
-Group 2 (Specific Instructions)
-
0 .6 -
0
0.2
l
Puzzle 1
0.
-4- Control Group
-0.4- Group 1 (Generic Instructions)
-*- Group 2 (Specific Instructions)
0.0
Puzzle
5
Figure 13. Effect of the
Instructions on the Rate of
Correct Answers, Section 1
Puzzle 1
Puzzle 4
Figure 14. Effect of the
Instructions on the Rate of
Correct Answers, Section 2.
Here we present the effect of the instructions on the rate of the ultimate correct
answers. The data is very similar to the effect of the instructions on the rate of
correct answers achieved on the first attempt. This says that the respondents are
equally likely to produce the ultimate correct answer as the correct answer on the
first attempt.
Page 51 of 79
The general instructions had a neutral effect on the performance of Group 1 in
Section 1 and hindered the performance of the same group in Section 2. The
negative effect of the Generic instructions in the Section 2 may be explained by
the fact that the Puzzle 4 was more difficult than the previous puzzles in this
series and the instructions were not readily available. So, the respondents in this
group found themselves in a situation when they had to resort to the selfdiscovery mode, which they have not practiced.
The effect of the instructions on Group 2 is more complex. While the specific
instructions hindered the performance of this group in Section 1, they turned out
beneficial in Section 2.
One of the reasons why the respondents practicing self-discovery were able to
outperform those who received instructions, could be due to the fact that they
developed a better thinking flexibility. In the first puzzle the respondents in the
control group had to examine multiple solution paths before finding the one that
yielded the correct result. This more extended search served two purposes: open
the scope of potential solutions and increase the expertise by trying the approach
on other solution paths.
Page 52 of 79
Effect of the Instructions on Number of Attempts
-
2.0
-- - -4- Control Group
-1U-Group 1 (Generic Instructions)
1.
0 1.5
0 .5
-ar- Group 2 (Specific Instructions)
--- - - - - - - - - - - _ _ _ _ _-
0
Z 0 .0
P
Puzzle 1
j2.5
E
0 2.0
1
:N
1.5
- -
1.0
-
- - -
-
- - - - - - -
-
--
--- Control Group
0.5 -Group 1 (Generic Instructions)
-*- Group 2 (Specific Instructions)
Puzzle 5
Figure 15. Effect of the
Instructions on Number of
Attempts, Section 1
- - - - - - - - -
-
2.5 --
E
-
(
Puzzle 1
Puzzle 4
Figure 16. Effect of the
Instructions on Number of
Correct Answers, Section 2
If the number of attempts is considered together with the time, it can be seen that
the respondents in the experimental Groups 1 and 2 spend progressively less time
on generating a solution and trying it out. Unfortunately for them, their rate of
finding the correct solution is not as high as for those in the Control Group and
they need to try again. It appears that the instructions promote the trail and error
mode of operation, while the respondents in the Control Group "aim and shoot"
more precisely.
Page 53 of 79
Effect of the Instructions on Number of Give Ups
3.0-
3.0-
2.5 -- - - - - - - - - -
- -A - -
-
CL
2.5 - -4-Control Group
-a- Group 1 (Generic Instructions)
2~ 2.00
*, Group 2 (Specific Instructions)
*
1.0
0 0.50.0
~1.5
N
-
--
_
_
_
_
_
_
_
---
-4- Control Group
-- Group 1 (Generic Instructions)
-A-- Group 2 (Specific Instructions)
1.5 -
0
-------------
-
- --
--
-
2.0 -- - - ~1.5-
0.5---------
--
0.0
Puzzle 1
Puzzle 5
Figure 17. Effect of the
Instructions on Number of
Give Ups, Section 1
Puzzle 1
Puzzle 4
Figure 18. Effect of the
Instructions on Number of Give
Ups, Section 2
The respondents in the Groups 1 and 2 are also more likely to give up.
It
suggests that the reliance on the tool adversely affects the persistence of the
respondents.
May need a button to invoke the instructions and stop time or provide a hard
copy of the instructions for continuous reference in the course of the experiment.
Effects of Instructional Types on Long-Term Memory Retention
Another effect of instructional type - retention in memory - was not examined in
this thesis. Although hypothesis can be made that the self-discovery will lead to a
better retention in memory since those practicing it discover the underlying
principle of the puzzle, the time frame of the thesis project is not sufficient to
conduct a time-delayed experiment to assess the long-term memory retention.
Page 54 of 79
Chapter 7
CONCLUSIONS
As hypothesized in Chapter 3, the instructions clearly affected the performance
of the two Experimental Groups with respect to the Control Group.
While the
trends are not entirely consistent for all the experiments, some important
conclusions can still be drawn from this experiment.
Parameters measured in the experiment, such as time and rate of correct answers
give a good indication of the quality of learning transfer, while the other
parameters - namely the number of attempts and give ups - are good indicators
of the persistency and determination on the part of the respondents. Instructions
clearly affected the respondents on both dimensions.
Close examination of the measured parameters reveals that not all the variables
are truly independent. In fact, the rate of correct answers (ROC) is proportional
to time spent and inversely proportional to the number of attempts and give ups.
ROC -
Time
Number of Attempts * Give ups
The individuals in the Control Group, who received no instructions and thus
were forced to practice self-discovery, on the average spent more time, exercised
fewer attempts and gave up less. This demonstrates a higher level of commitment
and the drive to deliver the correct solution on the first attempt (drive for quality
of result).
Page 55 of 79
The individuals receiving instructions, spent less time developing the solutions,
but tried more. Unfortunately, their level of commitment suffered as well (more
give ups), leading to worse overall result.
Instructions narrow the scope of the solution space by focusing the effort and
steering the respondents away from the erroneous directions.
Solution
Solution
0
0
Instructions
Problem
Problem
Figure 19. Solution Process
for Self-Discovery
Figure 20. Solution Process
with Instructions
However, instead of thinking about how to solve the problem, the respondents
think about how to apply the instructions.
The respondent engages in the
iterative process of understanding the instructions and figuring out how the
instructions relate to the problem at hand. In the case of generic instructions,
more iterations (solution attempts) may be required to connect the consolidated
and more abstract knowledge in the instructions to the problem than in the case
of specific instructions, where the instructions are so simple that they serve to
provide a quick glimpse or insight into the problem. For the relatively simple
puzzles presented in the survey, the simple and quick specific instructions may be
a better approach. However, for a more difficult problem, generic instruction will
Page 56 of 79
probably be a better method, as they will serve to both educate the user and assist
with finding the solution.
If the instructions do not fit the problem formulation well, or are not transparent
to the respondent, they become a liability.
The respondent tries applying the
instructions, giving it less thought then probably needed, receives an incorrect
result and gives up. He or she might be thinking: "I was given this tool and told
that it should work. I tried it several times and it obviously does not work. I give
up". The respondents in the Experimental Groups are not conditioned to think
through the puzzles in the same way as those in the Control Group, who received
no instructions and were required to deduce the solutions from the very
beginning.
Instructions stifled creativity in this experiment and adversely affected the
problem solving skills of the respondents.
Page 57 of 79
BIBLIOGRAPHY
1. Altshuller G.S., Creativi as an Exact Science, Gordon and Breach Science
Publishers Inc., 1984
2. Anderson J.R., Farrell R., Sauer R., Learningto Programin LISP, Cognitive
Science, Vol., pp. 87-129, 1984
3.
Blissett S.E., McGrath R.E., The relationship belween creativit and interpersonal
problem-solving skills in adults, Journal of Creative Behavior, 1996, pp 173182
4. Bransford J.D., Brown A.L., Cocking R.R., How People Learn, National
Research Council, 1999
5.
Cohen W.M., Levinthal D.A., Absorptive Capacit:A New Perspective on
Learningand Innovation, Administrative Science Quarterly, Vol. 2, pp. 128152, 1990
6.
Cormier, S. & Hagman, J., Transfer ofLearning. San Diego, CA: Academic
Press, 1987.
7.
Csikszentmihalyi M., Creativio:Flow and the Psychology of Discovery and
Invention, HarperCollins, New York, NY 1996
8.
Drabkin S., Enhancingcreativit when solving contradictogtechnicalproblem,
Journal of Professional Issues In Engineering Education and Practice,
Apr 1996, pp 78-82
9.
Ellis H.C., The TransferofLearning, The Macmillan Company, NY, 1965
10. Feldman D.H., Csikszentmihalyi M., Gardner H., Changing the World: A
Frameworkforthe Study of Creativiy, Praeger, Westport, CT, 1994
11. Fontenot N.A., Effects of trainingin creativit and creativeproblemfindingupon
businesspeople, Journal of Social Psychology, Feb 1993, pp 11-22
12. Fulton J., MENSA The Genius Test, Carlton Books, London, UK, 1999
13. Gardner H., Creating Minds
14. Grose, R. & Bimey, R., Transfer Of Learning. Princeton, NJ: Van Nostrand,
1963
15. Haskell, R., ReengineeringCorporate Training Intellectual Capitaland Trans/erof
Iearning, Quorum Books, Westport, CT, 1998
Page 58 of 79
16. Horowitz, R., Maimon, 0., CreativeDesign Methodology and the SIT Metohd,
ASME Design Engineering Technical Conference, Sacramento, CA, 1997
17. Leonard D, Sensiper S, The Role of Tacit Knowledge in group innovation,
California Management Review, 40 (3): 112-+, Spr 1998
18. Marakas G.M., Elam J.J., Creativit enhancement in problem solving: Through
software orprocess?, Management Science, Aug 1997, pp 1136-1146
19. Niedelman, M., Problem Solving and Transfer, Journal of Learning
Disabilities, Vol. 24, 1991
20. Pinker S., How the Mind Works, W.W. Norton & Company, New York,
NY, 1997
21. Roget's IH: The New Thesaurus, Third Edition by the Editors of the
American Heritage® Dictionary, Houghton Mifflin Company, 1995
22. Ruscio A.M., Amabile T.M., Effects of instructionalsyle on problem-solving
creativity, Creativity Research Journal, 1999, pp 251-266
23. Sickafus, E., Unfied StructuredInventive Thinking, Ntelleck, LLC, 1998
24. Simon H.A., The Sciences oftheArtzficial, The MIT Press, Cambridge, MA,
1996
25. Smith P.G., Reinertsen D. G., DevelopingProductsin Hafthe Time, John
Wiley & Sons, 1998
26. Stefan, C., StructuredInventive Thinking,
http://www.srl.ford.com/sitteam/sit.htm
27. Steiner C, A rolefor individuality andmysteg in "managing"change, Journal of
Organizational Change, 14(2): 150-167, 2001
28. Treffinger D.J., Creativeproblem-solving- overview and educationalimplication,
Educational Psychology Review, Sep 1995, pp 301-312
29. TRIZ Journal, www.triz-journal.com
30. www.Wordsmyth.net
31. Zander, Rosamund Stone and Zander, Benjamin. The Art ofPossibiliy,
Harvard Business School Press, Boston, MA, 2000
Page 59 of 79
Appendix
-
Complete Results
of the Experiment
ANOVA Table for RT
Inclusion criteria: Question IS 'Typel Q1" from Ben.Stat
DF
Cond
Sum of Squares
Mean Square
2
14756.813
7378.406
Residual 181
750102.500
9260.525
F-Value
P-Value
.797
Means Table for RT
Effect: Cond
Inclusion criteria: Question IS "Type1Q1" from Ben.Stat
Count
Mean
Std. Dev.
Std. Err.
Generic
24
121.583
144.144
29.423
Self-discovery
33
93.298
76.443
13.307
Specific
27
91.111
57.253
11.018
Fisher's PLSD for RT
%
Effect: Cond
Significance Level: 5
Inclusion criteria: Question IS "Type1Q1" from Ben.Stat
Mean Diff.
Crit. Diff.
P-Value
Generic, Self-discovery
28.285
51.366
.2765
Generic, Specific
30.472
53.716
.2623
2.187
49.687
.9304
Self-discovery, Specific
Interaction Bar Plot for RT
Effect: Cond
Inclusion criteria: Question IS "Typel Q1" from Ben.Stat
140
120
100
(D
80
~'60
40
CeU
20
0
Generic
Self-discovery
Cell
Specific
Page 60 of 79
.4543
Lambda Power
1.594
.176
ANOVA Table for Correct1?
Inclusion criteria: Question IS "Type1Q1" from Ben.Stat
DF
Cond
Sum of Squares
2
Residual
Mean Square
.549 1
15.201 1
1811
F-Value
P-Value
.275 1
1.463 1
.188 1
I
Means Table for Correcti?
Effect: Cond
Inclusion criteria: Question IS "Type1Q1" from Ben.Stat
Count
Mean
Std. Dev.
Std. Err.
Generic
24
.625
.495
.101
Self-discovery
33
.818
.392
.068
Specific
27
.7781
.424
.082
Fisher's PLSD for Correcti?
Effect: Cond
%
Significance Level: 5
Inclusion criteria: Question IS "Type1Q1" from Ben.Stat
Mean Diff.
Generic, Self-discovery
Generic, Specific
Self-discovery, Specific
Crit. Diff.
P-Value
-.193
.231
.1003
-.153
.242
.2123
.224
.7202
.040
Interaction Bar Plot for Correct1?
Effect: Cond
Inclusion criteria: Question IS "Type1Q1" from Ben.Stat
.9
.8
.7
.6
.5
( .4
.3
.2
.1
0
Generic
Self-discovery
Cell
Specific
Page 61 of 79
.2375
Lambda
1
I
2.927
Power
1
I
.293
I
ANOVA Table for Giveup?
Inclusion criteria: Question IS 'ypelQl" from Ben.Stat
DF
Cond
Sum of Squares
2
Residual
Mean Square
1 81 1
F-Value
.012 1
.024 1
1.928 1
.512
P-Value
1
II
.024 1
Means Table for Giveup?
Effect: Cond
Inclusion criteria: Question IS "Type1Q1" from Ben.Stat
Count Mean Std. Dev.
Std. Err.
24
33
27
Generic
Self-discovery
Specific
.042
.030
0.000
.204
.174
0.000
.042
.030
0.000
Fisher's PLSD for Giveup?
Effect: Cond
%
Significance Level: 5
Inclusion criteria: Question IS "Type1Q1" from Ben.Stat
Mean Diff.
Generic, Self-discovery
.011
Crit. Diff.
P-Value
.082
.7844
Generic, Specific
.042
.086
.3386
Self-discovery, Specific
.030
.080
.4513
Interaction Bar Plot for Giveup?
Effect: Cond
Inclusion criteria: Question IS 'ype1Q1" from Ben.Stat
.045
-
.035
-
.04
-
-
.03
.025
-
.01
.005
-
.015
-
-
> .02
0
-
U
Generic
Self-discovery
Specific
Cell
Page 62 of 79
.6015
Lambda Power
1
II
1.023
1
II
.128I1
.128
II
ANOVA Table for # of attempt
Inclusion criteria: Question IS "Type1Q1" from Ben.Stat
Cond
1
DF
Sum of Squares
2
.493
Residual 1811
Mean Square
.247
41.650
1
P-Value
F-Value
.479 1
.514 1
I
Means Table for # of attempt
Effect: Cond
Inclusion criteria: Question IS "Type1Q1" from Ben.Stat
Count
Mean
Std. Dev.
Std. Err.
Generic
24
1.333
1.049
.214
Self-discovery
33
1.152
.566
.098
Specific
27
1.185
.483
.093
Fisher's PLSD for # of attempt
Effect: Cond
%
Significance Level: 5
Inclusion criteria: Question IS "Type1Q1" from Ben.Stat
Mean Diff.
Crit. Diff.
P-Value
Generic, Self-discovery
.182
.383
.3474
Generic, Specific
.148
.400
.4636
-.034
.370
.8569
Self-discovery, Specific
Interaction Bar Plot for # of attempt
Effect: Cond
Inclusion criteria: Question IS "Type1Qi" from Ben.Stat
1.4
1.2
1
.8
0
.6
.4
.2
0
Generic
Self-discovery
Cell
Specific
Page 63 of 79
.6209
Lambda Power
1
I
.959 1
.123 1
I I
ANOVA Table for Final Correct?
Inclusion criteria: Question IS "Type1Q1" from Ben.Stat
DF
Sum of Squares
Cond
I 2I1.076 I
Residual
81 19.912
Mean Square F-Value
.538
1.122
I
4.395
P-Value
I
.0154
1
Effect: Cond
Inclusion criteria: Question IS "Type1Q1" from Ben.Stat
Mean Std. Dev.
Std. Err.
Generic
24
.667
.482
.098
Self-discovery
33
.909
.292
.051
Specific
27
.926
.267
.051
Fisher's PLSD for Final Correct?
%
Effect: Cond
Significance Level: 5
Inclusion criteria: Question IS "Type1Q1" from Ben.Stat
Mean Diff.
Generic, Self-discovery
Generic, Specific
Self-discovery, Specific
Crit. Diff.
P-Value
-.242
.187
.0116
S
-.259
.195
.0099
S
-.017
.181
.8533
Interaction Bar Plot for Final Correct?
Effect: Cond
Inclusion criteria: Question IS "Type1Q1" from Ben.Stat
-
.2
-
-
-
.4
.3
.1
-
o
-
Q .6
2.5
-
-
.8
.7
-
1.9
0Generic
8.790
11
Means Table for Final Correct?
Count
Lambda Power
Specific
Page 64 of 79
.74
ANOVA Table for RT
Inclusion criteria: Question IS "TypelQ5" from Ben.Stat
DF Sum of Squares
Mean Square F-Value
Cond
43460.505
Residual 81
961258.279
11867.386
I2 I
P-Value
I21730.253 I1.831 I.1668 I3.662 I.359
Means Table for RT
Effect: Cond
Inclusion criteria: Question IS 'ypelQ5" from Ben.Stat
Count
Mean
Std. Dev.
Std. Err.
Generic
24
59.653
57.867
11.812
Self-discovery
33
98.456
162.261
28.246
Specific
27
47.043
40.061
7.710
Fisher's PLSD for RT
Effect: Cond
%
Significance Level: 5
Inclusion criteria: Question IS 'ypelQ5" from Ben.Stat
Mean Diff.
Generic, Self-discovery
Crit. Diff.
P-Value
-38.803
58.148
.1880
Generic, Specific
12.610
60.808
.6810
Self-discovery, Specific
51.413
56.247
.0727
Interaction Bar Plot for RT
Effect: Cond
Inclusion criteria: Question IS "TypelQ5" from Ben.Stat
-
40
-
20
-
60
-
80
2
-
120
100
Lambda Power
0Generic
Self-discovery
Specific
Cell
Page 65 of 79
ANOVA Table for Correct1?
Inclusion criteria: Question IS "TypelQ5" from Ben.Stat
DF
Cond
Sum of Squares
2
Mean Square
.669 1
20.034 1
Residual 1811
F-Value
.334 1
P-Value
1.352 1
I
.247 1
Means Table for Correct1?
Effect: Cond
Inclusion criteria: Question IS "TypelQ5" from Ben.Stat
Count
Mean
Std. Dev.
Self-discovery
Std. Err.
I.. 33
.545
.506
.088
27
.407
.501
.096
24 1.3331
Generic
Specific
.482 1
.0981
Fisher's PLSD for Correct1?
Effect: Cond
%
Significance Level: 5
Inclusion criteria: Question IS "Type1Q5" from Ben.Stat
Mean Diff.
Generic, Self-discovery
-.212
Generic, Specific
Crit. Diff.
P-Value
.265
.1158
-.074
.278
.5969
.138
.257
.2879
Self-discovery, Specific
Interaction Bar Plot for Correct1?
Effect: Cond
Inclusion criteria: Question IS 'ypelQ5" from Ben.Stat
.5
-
.6
S.4-
.2
-
.1
-
2.3-
-
0
Generic
Self-discovery
Specific
Cell
Page 66 of 79
Lambda Power
.2645 1
I
2.704
.273
I
I
ANOVA Table for Giveup?
Inclusion criteria: Question IS "TypelQ5" from Ben.Stat
DF
Cond
Mean Square
Sum of Squares
1 .478
.249
.498 1
1
2
Residual 1811
.2341
I
.168 1
13.645
Means Table for Giveup?
Effect: Cond
Inclusion criteria: Question IS "TypelQ5" from Ben.Stat
Count
Mean
Generic
24
.250
Self-discovery
33
Specific
27
Std. Dev.
Std. Err.
.442
.090
.121
.331
.058
.296
.465
.090
Fisher's PLSD for Giveup?
Effect: Cond
%
Significance Level: 5
Inclusion criteria: Question IS "TypelQ5" from Ben.Stat
Mean Diff.
Crit. Diff.
P-Value
.2456
.129
.219
Generic, Specific
-.046
.229
.6887
Self-discovery, Specific
-.175
.212
.1041
Generic, Self-discovery
Interaction Bar Plot for Giveup?
Effect: Cond
Inclusion criteria: Question IS "TypelQ5" from Ben.Stat
.3
-
.25
-
.35
a) .15
.1
.05
0Generic
Self-discovery
Cell
Specific
Page 67 of 79
Lambda
P-Value
F-Value
1
I
Power
2.957 1
I
.296 1
I
ANOVA Table for # of attempt
Inclusion criteria: Question IS "TypelQ5" from Ben.Stat
Cond
DF
Sum of Squares
2
3.812
Residual 1811
Mean Square
F-Value
P-Value
.913
.4055
1.906
169.140
1
I
2.088
2.088
Means Table for # of attempt
Effect: Cond
Inclusion criteria: Question IS "TypelQ5" from Ben.Stat
Count
Mean
Std. Dev.
Std. Err.
Generic
24
1.958
2.010
.410
Self-discovery
33
1.545
.754
.131
Specific
27
2.000
1.494
.287
Fisher's PLSD for # of attempt
Effect: Cond
%
Significance Level: 5
Inclusion criteria: Question IS "TypelQ5" from Ben.Stat
Mean Diff.
Crit. Diff.
.413
.771
.2900
Generic, Specific
-. 042
.807
.9184
Self-discovery, Specific
-.455
.746
.2290
Generic, Self-discovery
P-Value
Interaction Bar Plot for # of attempt
Effect: Cond
Inclusion criteria: Question IS "TypelQ5" from Ben.Stat
2.25
2
-
1.5
1.25
-
1.75
-
.5
-
.25
-
1-
.75
0Generic
Self-discovery
Cell
Specific
Page 68 of 79
Lambda
1.826
I
Power
.196
ANOVA Table for Final Correct?
Inclusion criteria: Question IS "TypelQ5" from Ben.Stat
DF Sum of Squares
Mean Square F-Value P-Value
Cond
2
I .12F,~
.835 I
..417 I 2 .131
Residual 811
15.867 I
.196 1
I
Means Table for Final Correct?
Effect: Cond
Inclusion criteria: Question IS "TypelQ5" from Ben.Stat
Count Mean Std. Dev. Std. Err.
Generic
24
.625
.495
.101
Self-discovery
33
.848
.364
.063
Specific
27
.667
.480
.092
Fisher's PLSD for Final Correct?
%
Effect: Cond
Significance Level: 5
Inclusion criteria: Question IS "Type1Q5" from Ben.Stat
Generic, Self-discovery
Generic, Specific
Self-discovery, Specific
Mean Diff.
-.223
-.042
.182
Crit. Diff.
.236
.247
.229
P-Value
.0634
.7381
.1173
Interaction Bar Plot for Final Correct?
Effect: Cond
Inclusion criteria: Question IS "Type1Q5" from Ben.Stat
.9
.8
.7
c .6
~5
a).4
U.3
.2
.1
0
Generic
Self-discovery
Cell
Specific
Page 69 of 79
Lambda Power
I
42~i2
..I1AI
I-I--I
I
412
I
ANOVA Table for RT
Inclusion criteria: Question IS "Type2Q1" from Ben.Stat
DF
I2 I
Sum of Squares
Cond
Residual
81
875054.624
25456.488
Mean Square F-Value
P-Value
I12728.244 I1.178 I.3131 I2.356 I.242
10803.144
Means Table for RT
Effect: Cond
Inclusion criteria: Question IS "Type2Q1" from Ben.Stat
Count
Mean Std. Dev.
Std. Err.
Generic
24 65.007
192.719
39.339
Self-discovery
33 24.157
18.589
3.236
Specific
27
30.227
19.381
3.730
Fisher's PLSD for RT
Effect: Cond
%
Significance Level: 5
Inclusion criteria: Question IS "Type2Q1" from Ben.Stat
Mean Diff. Crit. Diff.
P-Value
Generic, Self-discovery
40.850
55.480
.1468
Generic, Specific
34.781
58.017
.2364
Self-discovery, Specific
-6.070
53.666
.8225
Interaction Bar Plot for RT
Effect: Cond
Inclusion criteria: Question IS 'Type2Q1" from Ben.Stat
70
60
50
40
0D 30
20
10
0
Generic
Lambda Power
Self-discovery
Cell
Specific
Page 70 of 79
ANOVA Table for Correct1?
Inclusion criteria: Question IS "Type2Q1" from Ben.Stat
Cond
DF
Sum of Squares
2
.002
Residual 1 81 1
Mean Square
F-Value
.001
2.891
P-Value
~.72
.026
.0361
1
Means Table for Correcti?
Effect: Cond
Inclusion criteria: Question IS "Type2Q1"from Ben.Stat
Count
Mean Std. Dev.
Std. Err.
Generic
24
.958
.204
.042
Self-discovery
33
.970
.174
.030
Specific
27
.963
.192
.037
Fisher's PLSD for Correct1?
%
Effect: Cond
Significance Level: 5
Inclusion criteria: Question IS 'ype2Q1" from Ben.Stat
Mean Diff.
Crit. Diff.
P-Value
Generic, Self-discovery
-.011
.101
.8232
Generic, Specific
-.005
.105
.9306
.007
.098
.8911
,
Self-discovery, Specific
.6
.2
-
o 4-
0
-
(
-
.8
-
Interaction Bar Plot for Correcti?
Effect: Cond
Inclusion criteria: Question IS "Type2Q1" from Ben.Stat
Generic
Self-discovery
Specific
Cell
Page 71 of 79
Lambda
52
Power
.054
Cond
I
.030
2
Residual 181 1
P-Value
I
.015 I1.258
.
II
.012I
.012
.958 |
..2898
Means Table for Giveup?
Effect: Cond
Inclusion criteria: Question IS "Type2Q1" from Ben.Stat
Count
Mean
Std. Dev.
24
.042
33 0.000
27 10.000 1
Generic
Self-discovery
Specific
[
Std. Err.
.204
0.000
0.000 1
.042
0.000
0.000
Fisher's PLSD for Giveup?
Effect: Cond
%
Significance Level: 5
Inclusion criteria: Question IS "Type2Q1" from Ben.Stat
Mean Diff.
Crit. Diff.
P-Value
Generic, Self-discovery
.042
.058
.1572
Generic, Specific
.042
.061
.1759
0.000
.056
Self-discovery, Specific
Interaction Bar Plot for Giveup?
Effect: Cond
Inclusion criteria: Question IS "Type2Q1" from Ben.Stat
-
-
.03
.015
-
& .02
-
.025
-
.04
.035
-
.045
.005
-
i.12
ANOVA Table for Giveup?
Inclusion criteria: Question IS "Type2Q1" from Ben.Stat
DF Sum of Squares
Mean Square F-Value
0
Generic
Self-discovery
Cell
Specific
Page 72 of 79
Lambda Power
256
I2516
II
II
II
ANOVA Table for # of attempt
Inclusion criteria: Question IS "Type2Q1" from Ben.Stat
Cond
DF
Sum of Squares
2
.074
Residual 181 1
Mean Square
F-Value
.037
3.879
P-Value
Lambda
.4671
1.537
.768
.048
1
I
I
Means Table for # of attempt
Effect: Cond
Inclusion criteria: Question IS "Type2Q1" from Ben.Stat
Count
Generic
Self-discovery
I.
Specific
Mean
Std. Dev.
Std. Err.
24
1.000
0.000
33
1.061
.348
.061
27
1.000
0.000
0.000
0.000 1
Fisher's PLSD for # of attempt
Effect: Cond
%
Significance Level: 5
Inclusion criteria: Question IS "Type2Q1" from Ben.Stat
Mean Diff.
Generic, Self-discovery
Generic, Specific
Self-discovery, Specific
L
Crit. Diff.
P-Value
-.061
.117
.3050
0.000
.122
2
.061
.113
.2890
Interaction Bar Plot for # of attempt
Effect: Cond
Inclusion criteria: Question IS 'Type2Q1" from Ben.Stat
1.2
I
a .8
0)
.6
U
.4
.2
0
Generic
Self-discovery
Specific
Cell
Page 73 of 79
Power
.171
II
II
ANOVA Table for Final Correct?
Inclusion criteria: Question IS "Type2Q"from Ben.Stat
Cond
DF
Sum of Squares
2
.031
Residual 181 1
Mean Square
F-Value
.016
1.921
1
P-Value
.655
.5220 1
.I
.024 1
Means Table for Final Correct?
Effect: Cond
Inclusion criteria: Question IS "Type2Q1" from Ben.Stat
Count
Generic
Self-discovery
L
Specific
Mean
Std. Dev.
Std. Err.
.958
.204
33
1.000
0.000
0.000
27
.963
.192
.037
241
.042 1
Fisher's PLSD for Final Correct?
Effect: Cond
%
Significance Level: 5
Inclusion criteria: Question IS "Type2Q1" from Ben.Stat
Mean Diff. Crit. Diff.
P-Value
Generic, Self-discovery
-.042
.082
.3162
Generic, Specific
Self-discovery, Specific
-.005
.037
.086
.080
.9149
.3568
Interaction Bar Plot for Final Correct?
Effect: Cond
Inclusion criteria: Question IS "Type2Q1"from Ben.Stat
1.2
.8
-
-
1
.6U-
.4.20Generic
Self-discovery
Specific
Cell
Page 74 of 79
Lambda Power
I
1.310
.152
I
I
ANOVA Table for RT
Inclusion criteria: Question IS "Type2Q4" from Ben.Stat
DF
Cond
Sum of Squares
2
Residual 1 81 1
Mean Square
14114.446
7057.223
1018775.104
12577.470
F-Value
.561
P-Value
1
I
Means Table for RT
Effect: Cond
Inclusion criteria: Question IS 'ype2Q4" from Ben.Stat
Count
Mean
Std. Dev.
Std. Err.
24
121.099
134.434
1
27.441
Self-discovery
33
95.673
97.965
1
17.053
Specific
27 1 123.115
1 106.699
1
20.534
,
Generic
Fisher's PLSD for RT
Effect: Cond
%
Significance Level: 5
Inclusion criteria: Question IS "Type2Q4" from Ben.Stat
Mean Diff.
Crit. Diff.
P-Value
Generic, Self-discovery
25.426
59.863
Generic, Specific
-2.016
62.601
.9491
-27.442
57.905
.3485
Self-discovery, Specific
.4006
Interaction Bar Plot for RT
Effect: Cond
Inclusion criteria: Question IS "Type2Q4" from Ben.Stat
140
120
100
C
80
CO)
60
40
20
0
Generic
Self-discovery
Specific
Cell
Page 75 of 79
.5728
Lambda Power
1
I
1.122
.137
I
I
ANOVA Table for Correct1?
Inclusion criteria: Question IS "Type2Q4" from Ben.Stat
DF
Cond
Sum of Squares
Mean Square
.027
2
Residual 181 1
16.675
F-Value
.014
1
P-Value
.066
.9359
I
.206 1
Means Table for Correcti?
Effect: Cond
Inclusion criteria: Question IS "Type2Q4" from Ben.Stat
Count Mean Std. Dev.
Std. Err.
Generic
24
.250
.442
.090
Self-discovery
33
.273
.452
.079
Specific
27
.296
.465
.090
Fisher's PLSD for Correct1?
Effect: Cond
%
Significance Level: 5
Inclusion criteria: Question IS "Type2Q4" from Ben.Stat
Mean Diff.
Crit. Diff.
P-Value
Generic, Self-discovery
-.023
.242
.8524
Generic, Specific
-.046
.253
.7170
Self-discovery, Specific
-.024
.234
.8418
Interaction Bar Plot for Correcti?
Effect: Cond
Inclusion criteria: Question IS "Type2Q4" from Ben.Stat
.25
-
.3
-
.35
.~2
Q .15
.05
-
.1-
0Generic
Self-discovery
Specific
Cell
Page 76 of 79
Lambda Power
1
I
.133 1
II
.060 1
ANOVA Table for Giveup?
Inclusion criteria: Question IS "Type2Q4" from Ben.Stat
DF Sum of Squares
Mean Square F-Value
Cond
1
2
Residual 1811
1.319
1
.660 1
17.002
1
.210
P-Value
3.143 1
I
I
I
Means Table for Giveup?
Effect: Cond
Inclusion criteria: Question IS "Type2Q4" from Ben.Stat
Count Mean Std. Dev. Std. Err.
Generic
24
.458
.509
.104
Self-discovery
33
.364
.489
.085
Specific
27
.148
.362
.070
%
Fisher's PLSD for Giveup?
Effect: Cond
Significance Level: 5
Inclusion criteria: Question IS "Type2Q4"from Ben.Stat
Mean Diff. Crit. Diff.
P-Value
Generic, Self-discovery
.095
.245
.4433
Generic, Specific
.310
.256
.0181
Self-discovery, Specific
.215
.237
.0736
Interaction Bar Plot for Giveup?
Effect: Cond
Inclusion criteria: Question IS "Type2Q4" from Ben.Stat
.5
.45
.4
.35
( .3
2.25
o
.2
.15
.1
.05
0
Generic
Self-discovery
Cell
Specific
Page 77 of 79
Lambda
.0485 1
S
6.285
Power
1
I
.581 1
_j
ANOVA Table for # of attempt
Inclusion criteria: Question IS "Type2Q4" from Ben.Stat
Cond
DF
Sum of Squares
2
42.738
Residual I181 1
Mean Sauare
F-Value
21.369
1
1552.214
1.115
P-Value
P-Value
.3329
19.163
Means Table for # of attempt
Effect: Cond
Inclusion criteria: Question IS "Type2Q4" from Ben.Stat
Count
Mean Std. Dev.
Std. Err.
Generic
24 3.542
6.467
1.320
Self-discovery
33
1.788
1.139
.198
Specific
27
2.519
4.594
.884
Fisher's PLSD for # of attempt
Effect: Cond
%
Significance Level: 5
Inclusion criteria: Question IS "Type2Q4" from Ben.Stat
Mean Diff.
Crit. Diff.
P-Value
Generic, Self-discovery
1.754
2.337
.1392
Generic, Specific
1.023
2.444
.4072
Self-discovery, Specific
-.731
2.260
.5219
Interaction Bar Plot for # of attempt
Effect: Cond
Inclusion criteria: Question IS "Type2Q4" from Ben.Stat
4
3.5
3
2.5
2
U1.5
1
.5
0
Generic
Self-discovery
Specific
Cell
Page 78 of 79
Power
Lambda
Lambda Power
2.230
.231
ANOVA Table for Final Correct?
Inclusion criteria: Question IS "Type2Q4" from Ben.Stat
DF
Cond
Sum of Squares
2
Mean Square
.817 1
20.076 1
Residual 181 1
F-Value
.409 1 1.648
P-Value
I
.248 1
II
Means Table for Final Correct?
Effect: Cond
Inclusion criteria: Question IS "Type2Q4" from Ben.Stat
Count
Mean
Std. Dev.
Std. Err.
Generic
24
.417
.504
.103
Self-discovery
33
.515
.508
.088
Specific
27
.667
.480
.092
Fisher's PLSD for Final Correct?
Effect: Cond
%
Significance Level: 5
Inclusion criteria: Question IS "Type2Q4"from Ben.Stat
Mean Diff. Crit. Diff.
P-Value
Generic, Self-discovery
-.098
.266
.4630
Generic, Specific
-.250
.278
.0772
Self-discovery, Specific
-.152
.257
.2443
Interaction Bar Plot for Final Correct?
Effect: Cond
Inclusion criteria: Question IS "Type2Q4"from Ben.Stat
.7
.6
.5
.
(D
.2
0
Generic
.1987
Specific
Page 79 of 79
Lambda Power
3.297 1
.326