An Empirical Question: How
Students Learn Economics?
Some Evidence from The JEE
(1969 to 2011)
Dr. Nürşen Albayrak Zanca
Associate Professor of Economics
University of the Incarnate Word
Department of Economics
San Antonio, Texas, 78209
zanca@uiwtx.edu
The Gulf Coast Economics Teaching Conference, October 27-28, 2011, New Orleans, Louisiana
OVERVIEW
 PROBLEM STATEMENT: Motivation & Objective
 EMPIRICAL EVIDENCE ON CLASSROOM
EXPERIMENTS: Do They Improve Learning?
 Literature from 1970s; 1980s; 1990s; and 2000s
 A META-ANALYSIS OF LECTURE-LESS
TEACHING APPROACH (The Journal of Economic
Education, 1969-2011)
 CONCLUDING REMARKS & COMMENTS
The Gulf Coast Economics Teaching Conference, October 27-28, 2011, New Orleans, Louisiana
PROBLEM STATEMENT: Motivation
& Objective
 The higher education in the U.S. has become a
mass process.
The U.S. Population
# of Enrollment
152 million (1950)
2.3 million (1950)
> 300 million (2010)
>16.9 million (2005)
 Financially, the reliance on large classes as the
major teaching device is very important to the
universities.
The Gulf Coast Economics Teaching Conference, October 27-28, 2011, New Orleans, Louisiana
PROBLEM STATEMENT: Motivation &
Objective
 As a result of this mass process,
lecture classes have become larger to
a point where there can be no
interaction between student and
teacher. In other words, students
become an entirely passive subject in
the learning process.
The Gulf Coast Economics Teaching Conference, October 27-28, 2011, New Orleans, Louisiana
Lecture-less vs. Lecture
 Education specialists agree that instructors
should use active learning.
 An important issue in economics education is
that the predominant method of teaching style
is traditional lecture method which provides
little opportunity for active learning.
 Lecture = “One-Way Talk” = “Chalk & Talk”
The Gulf Coast Economics Teaching Conference, October 27-28, 2011, New Orleans, Louisiana
PROBLEM STATEMENT: Motivation
& Objective
 An increasingly popular alternative to the
lecture-oriented ‘Chalk-and-Talk’ approach is
‘Classroom Experiments’
 Classroom experiments engages students in
active learning (away from passive learning).
 Some economists questioned whether the
choices that instructors make about how to
teach introductory economics make a
difference in learning of economics.
The Gulf Coast Economics Teaching Conference, October 27-28, 2011, New Orleans, Louisiana
A Wide Range of Other
EXPERIENTIAL Teaching Tools
 duration/length of the course
 classroom size
 student characteristics (cognitive skills, gender, seating, math skills, note taking skills)
 absenteeism
 use of algebra and graph
 use of newspaper articles
 use of sports; use of case studies
 Use of CAI (computer assisted instruction)
 use of computer technology and integrating various Internet tools
 use of music
 use of literature and drama, motion picture
 use of humor, using the Simpsons /animated cartoons
 use of acceptance speeches by the Nobel Laureates in economics and, and so on.
The Gulf Coast Economics Teaching Conference, October 27-28, 2011, New Orleans, Louisiana
METHODOLOGY
 Most frequently, researchers have conducted controlled tests, where
the relative effectiveness of any of the above listed experiential
learning style against traditional lecture was analyzed within a
multiple linear regression model.
 The dependent variable (a change measured as the difference in
post- and pre course TUCE scores) was controlled with a variety of
student- and instructor-level characteristics.
 Often, the independent variables included experimental group vs.
control group (captured with a dummy variable), cumulative GPA
(representing students aptitude), academic experience (credit hours
earned), business major, other student demographic characteristics,
such as sex, ethnicity, age, and so on.
The Gulf Coast Economics Teaching Conference, October 27-28, 2011, New Orleans, Louisiana
METHODOLOGY

The most common technique (Multiple Linear Regression)
Y = f ( S, PRE, X,  )
Y
S
adjusted post-TUCE examination score
teaching style 1 for the Experimental section
0 for the Control (lecture) section
PRE
pre-TUCE examination score
X
a vector of student characteristics
(Cumulative GPA, Academic Experience, Major, Sex, etc)

error term
 A test of differences between means (t-test)
The Gulf Coast Economics Teaching Conference, October 27-28, 2011, New Orleans, Louisiana
Scope & Limitations of the
Paper
 The primary aim of this paper is to provide a
literature review on classroom experiments.
 Articles focused on describing the actual
experiments are out of the scope of this paper.
 Articles in which instructors describe their
experiences using classroom experiments are
considered.
 The scope of the META-ANALYSIS is restricted
to literature review from the JEE (1969-2011)
The Gulf Coast Economics Teaching Conference, October 27-28, 2011, New Orleans, Louisiana
Objective of the Paper
 1st : The primary purpose is to determine to what degree
classroom experiments contribute to the learning of
economics.
 Is there any systematic evidence for classroom
experiments in learning of economics?
 Can students learn economics with classroom
experiments better than in the traditional lecture?
 2nd :To provide a literature review and a meta-analysis on
classroom experiments from the JEE since its inception
(Vol. 1, Fall 1969) up to date.
The Gulf Coast Economics Teaching Conference, October 27-28, 2011, New Orleans, Louisiana
Classroom Experiments:
Background
 Simply, teaching economics through the use of




experiments
A pedagogy that is derived from the field of
experimental economics.
Chamberlin used experimental economics to discover
Monopolistic Competition (1948)
Vernon L. Smith won Nobel Prize in Economics for
developing Experimental Economics (2002)
Vernon Smith is considered by many to be the father
of experimental economics.
The Gulf Coast Economics Teaching Conference, October 27-28, 2011, New Orleans, Louisiana
Trends in the number of publications in
experimental economics field (1948-1998)
 Books
 Textbooks
 WebPages
 Internet
Newsletter
Classroom
Experiments
 Academic
journals
 Conference
Papers
 Reports
 Unpublished
papers
The Gulf Coast Economics Teaching Conference, October 27-28, 2011, New Orleans, Louisiana
Empirical Evidence on Classroom
Experiments: Do They Improve
Learning?
 While there are many advocates of classroom experiments [Post
(1985); Millerd & Alastair (1987); Spencer & Van Eynde (1986); Walker
(1987); Friedman & Sunder (1994); Frank (1997); Gremmen & Potters
(1997); Holt (1999); Carlson & Skaggs (2000); Dickie (2000); Truscott et
al. (2000); Wilson & Nelson (2000); Barnett & Kriesel (2003); Jensen &
Owen (2003); McGuire & Zanca (2003); Emerson & Taylor (2004), Zanca
& McGuire (2005), and Dickie (2006)], there are also those who claim
classroom experiments have not been empirically any superior to the
chalk-and-talk method [McConnell & Lamphear (1969); Paden & Moyer
(1969); Danielsen & Stauffer (1972); Luker et al. (1975); Sharp, et al.
(1975); Wentford & Lewis (1975); Seigfried & Fels (1979); Frass (1982);
De Young (1993); Fels (1993); Williams & Walker (1993); Cardell et al.
(1996), Laury (1999); Yandell (1999); and Maxwell et al. (2005)]
The Gulf Coast Economics Teaching Conference, October 27-28, 2011, New Orleans, Louisiana
NEGATIVE Empirical Evidence
 No systematic evidence exists to support the
hypothesis that students participating in experiments
learn economics better than students getting lecture
treatment.
 Undergraduate instruction in economics is largely a
passive experience. Any opportunity that replaces
lectures with active participation would benefit
students.
 However, efficacy of classroom market experiments are
drawn from anecdotal evidence (positive remarks made
by students) and subjective analysis in supporting the
effectiveness of the experimental approach.
The Gulf Coast Economics Teaching Conference, October 27-28, 2011, New Orleans, Louisiana
POSITIVE Empirical Evidence
 Students exposed to the classroom experiments have
higher gains in the TUCE than their traditional lectureoriented (non-experimental) cohorts.
 The results were statistically significant.
 In conclusion, indeed, using classroom experiment
does increase student learning.
The Gulf Coast Economics Teaching Conference, October 27-28, 2011, New Orleans, Louisiana
Back to Question #1: Is there any
systematic evidence for classroom
experiments in learning of economics?
 Anecdotal Evidence
“Anything is better than a lecture!”
 “If it is good for me, it is good!”
 “This is what I do, and I like it!
So do my Students!”
 Empirical Evidence (i.e, statistical significance)
 Some say YES, and some say NO!

The Gulf Coast Economics Teaching Conference, October 27-28, 2011, New Orleans, Louisiana
What is a META-ANALYSIS?
 It is well acknowledged that it is not proper to
recommend an intervention or a treatment based on a
single study.
 Instead, it is advisable to examine the cumulative effect
of numerous studies before application. How to we learn
about the cumulative effects?
 The goal of the meta-analysis is accumulating
knowledge.
 Meta-analysis is the rubric used to describe quantitative
methods for combining evidence across studies.
The Gulf Coast Economics Teaching Conference, October 27-28, 2011, New Orleans, Louisiana
HOW TO CONDUCT
A META-ANALYSIS?
 STEP#1: Literature Search

The scope of this study is restricted to literature review from the
JEE. The JEE frequently offer original articles on innovative
teaching techniques in economics. (The JEE, Fall 1969, Volume
1, Number 1).
 STEP #2: Inclusion Criteria

Lecture-less vs. Traditional Lecture
 STEP #3: Describing, Classifying and Coding Research Studies
 STEP #4: Computation and Interpreting of Effect Sizes ()
The Gulf Coast Economics Teaching Conference, October 27-28, 2011, New Orleans, Louisiana
A META-ANALYSIS OF CLASSROOM
EXPERIMENTS (The JEE, 1969-2007)
 An effect size is defined as the strength of the relationship between
the independent variable and the dependent variable. For example, if
one is comparing an intervention group to a control group, the most
informative and straight forward measure of experimental effect size is
the mean difference between experimental and control groups divided
by pooled within-group standard deviation.
ˆ  (X E  X C ) / S

X

P  t 1 nE  1 nC 
1/ 2

S  nE 1S  nC 1S /nE  nC  2
2
P
2
E
2
C
The Gulf Coast Economics Teaching Conference, October 27-28, 2011, New Orleans, Louisiana
A META-ANALYSIS OF CLASSROOM
EXPERIMENTS (The JEE, 1969-2007)
 How should  (i.e., effect size) be
interpreted?
small effect
medium (moderate) effect
large effect
 = 0.2
 = 0.5
 = 0.8
The Gulf Coast Economics Teaching Conference, October 27-28, 2011, New Orleans, Louisiana
RESULTS OF THE
LITERATURE REVIEW
&
THE META-ANALYSIS
OF CLASSROOM
EXPERIMENTS
(The JEE, 1969-2007)
The Gulf Coast Economics Teaching Conference, October 27-28, 2011, New Orleans, Louisiana
RESULTS
 Economists have been active in the improvement of
their discipline.
 A great deal of work has been done in the
improvement of learning of economics.
 The effect of classroom experiments on student
learning has been widely investigated, but the results
have been inconclusive, and often contradictory.
 While the empirical evidence shows some support for
the classroom experiments over classical lecturing
methods, there is no consensus on this matter.
The Gulf Coast Economics Teaching Conference, October 27-28, 2011, New Orleans, Louisiana
RESULTS
 The current paper provides some cumulative evidence
on the impact of lecture-less teaching on student
learning in economics.
 The results of the meta-analysis indicate that the
average effect size for comparisons of lecture-less
(experiment group) vs. traditional lecture (control
group) instruction is less than 0.2.
 In other words, there is NO significant difference
between lecture-less vs. lecture style teaching on
students’ learning.
The Gulf Coast Economics Teaching Conference, October 27-28, 2011, New Orleans, Louisiana
RESULTS
 Evidence of this paper indicated that lecture-less
vs. lecture format of teaching does not have any
statistically significant difference in student
learning.
 In other words, students do not learn economics
better in experimental group vs. control group (i.e,
old-traditional-boring lecture!)
 As an advocate of classroom experiments, the
results of the meta-analysis was surprising for the
author!
The Gulf Coast Economics Teaching Conference, October 27-28, 2011, New Orleans, Louisiana
RESULTS
 The current paper provides some support
for classroom experiments being an
effective teaching tool, however, not
necessarily an effective learning method.
 Despite the inconclusive result on
learning of economics with lecture-less
class, there is a pattern for student as well
as instructor satisfaction with classroom
experiments.
The Gulf Coast Economics Teaching Conference, October 27-28, 2011, New Orleans, Louisiana
RESULTS
 A byproduct of classroom experiments is that it
renews instructors commitment to teaching and
students participation in active learning.
 The vast majority of students enrolled in
principles of microeconomics are not economics
majors, they generally have a limited aptitude for
microeconomic analysis.
 Such innovative, interesting techniques can
enhance the recruitment of economics majors
both to sustain our discipline, and to increase the
analytical skills of college students.
The Gulf Coast Economics Teaching Conference, October 27-28, 2011, New Orleans, Louisiana
RESULTS
 The reason to adopt of experiential teaching
techniques may vary from
 Making students active learners
 Attracting students into ECON
classes/discipline
 Making class less boring, less abstract and
more applicable
 Improve student evaluations
 Improving students’ learning is NOT the case!
The Gulf Coast Economics Teaching Conference, October 27-28, 2011, New Orleans, Louisiana
Lecture = “One-Way Talk” =
“Chalk & Talk”
 Some researchers argue may be students feel more comfortable
with the traditional lecture style, where the instructor feeds them
the information that they needed to know for the test.
 Some argue that the lecture method’s comparative advantages lie
in transmitting information.
 Lectures are often less effective than other teaching methods in
promoting independent thought, developing critical thinking skills
in students, and changing student attitudes.
 If one wants to achieve these latter objectives, one should be
aware of the limitations of one-way talk (and chalk), and modify the
lectures accordingly and/or supplement them with other teaching
devices.
The Gulf Coast Economics Teaching Conference, October 27-28, 2011, New Orleans, Louisiana
FUTURE RESEARCH
 Extend the Meta-Analysis to
Other Journals
 Conference papers
 Other form of discussion/academic papers
 Submit the findings of this paper to the JEE
TITLE: “Lecture vs. What? A Comparison of
Outcomes in Economics Learning?”

The Gulf Coast Economics Teaching Conference, October 27-28, 2011, New Orleans, Louisiana
FUTURE RESEARCH
 Many articles on this topic appear regularly in most
mainstream economics journals.
 Among those journals are American Economic Review,
Journal of Economic Education, Journal of Economic
Inquiry, Journal of Economic Perspective, Southern
Economics Journal, and American Economist, and
many others.
 Others, including conferences sponsored by the AEA,
and others (just like this one, the GCEA!) have sessions
on teaching of economics where innovative papers are
presented in this field.
The Gulf Coast Economics Teaching Conference, October 27-28, 2011, New Orleans, Louisiana
FUTURE RESEARCH
 Overall, the level of student interest and active learning
involvement component of classroom experiments
cannot be denied.
 There has not been enough evidence that these games
are improving students’ economics understanding.
 Economics educators should continue to search the
effect of experiential pedagogy on student learning.
 The author of this paper hopes that the paper has
created an appetite to the reader to learn more about
classroom experiments.
The Gulf Coast Economics Teaching Conference, October 27-28, 2011, New Orleans, Louisiana
PERSONAL ANECDOTAL EVIDENCE
(Positive remarks made by students)
n= 47 students, ECON 2302 MICRO
 Expernomics made economics much easier.
 It was very efficient getting the main principle across the






class.
It helped me to better understand and master the economic
concepts.
It helps learn better.
My only suggestion is to HAVE MORE games/experiments.
Other classes should adopt it, too.
Everyone participated b/c it involved food and money!
We tend to remember easier.
The Gulf Coast Economics Teaching Conference, October 27-28, 2011, New Orleans, Louisiana
Anecdotal Evidence….
On the first day of school, a teacher
was glancing over the roll when she
noticed a number after each student’s
name, such as 154, 136, or 142.
“Wow! Look at these IQs,” she said to
herself. “What a terrific class!”
The Gulf Coast Economics Teaching Conference, October 27-28, 2011, New Orleans, Louisiana
Anecdotal Evidence….
The teacher promptly determined to work
harder with this class than with any other
she ever had.
Throughout the year, she came up with
innovative lessons that she thought would
challenge the students, because she did
not want them to get bored with work that
was too easy.
The Gulf Coast Economics Teaching Conference, October 27-28, 2011, New Orleans, Louisiana
Anecdotal Evidence….
Her plan worked!
The class outperformed all the other
classes that she taught in the usual way.
Then, during the last quarter of the year,
she discovered what those numbers
after the students’ name really were:
their locker numbers.
The Gulf Coast Economics Teaching Conference, October 27-28, 2011, New Orleans, Louisiana
COMMENTS
&
QUESTIONS
Thank You!
The Gulf Coast Economics Teaching Conference, October 27-28, 2011, New Orleans, Louisiana