NEW AND ENDURING TOPICS
AND TRENDS IN ECONOMIC
EDUCATION RESEARCH
Michael Watts
Purdue University
2012 Korean Economic Education Association
International Conference, Seoul
2012 Council on Economic Education and
National Association of Economic Educators
Conference, Kansas City
1
SIMPLE THESES
1) The basic (“bread and butter”) set of questions asked in
economic education research over the past five decades has
not changed and is not likely to change.
2) Theoretical models and insights featured in the literature
are not dramatically different from what they were several
decades ago, either.
3) Instead, most new insights and findings have been based on
empirical research featuring various kinds of datasets, and
improvements in statistical and econometric methods for
analyzing data.
2
4) Future progress is also most likely to come from new and
better data sets and improved statistical methods,
5) A key challenge for economic education researchers is to
find and report results in ways that have meaningful
practical significance, not just statistical significance.
Doing that will attract more attention from other
economists, educators, policy makers, and the general
public. For example…..
In this presentation, I will touch only briefly on points 1- 4,
and focus on 5.
3
I. FIVE BASIC AND ENDURING
RESEARCH QUESTIONS IN ECONOMIC
EDUCATION RESEARCH
Question 1:
How can we teach economics more effectively, especially in
terms of student learning and interest in the subject, but
also in terms of constraints that are inescapable across all
fields of education – most notably tight budgets, limited
classroom time for instruction, and offering course and
instructional formats that provide more flexibility to
students and school administrators?
4
Question 2:
At the precollege level, how can we get more students to
take economics and (not or) personal/financial education
courses, and (not or) see more good economics instruction in
other courses (especially history, civics/government, math,
science/environmental studies, and languages/language
arts)?
5
Question 3:
How can we get more students to major or minor in economics at the
undergraduate level?
6
Question 4:
How can we better prepare Ph.D. students in economics to be better
teachers, and provide appropriate incentives and opportunities for
current professors to improve their teaching?
7
Question 5:
Can we teach economics in ways that make it more
memorable and useful to people making decisions as
consumers, workers, and citizens after they finish their
schooling, and can we show the general public and policy
makers in particular that economic and financial literacy
really do affect “real world” decisions people make all
of the time?
8
This list of core questions strikes me as self-evidently
enduring. But the answers to these questions can change
over time and vary across countries, based on changes or
differences in social, demographic, and technological
factors, as well as public policies.
9
II. TRENDS AND ENDURING PATTERNS
IN ECONOMIC EDUCATION RESEARCH
A. Data is Half (Maybe More) of the Battle
Key “Regular” Data Sources Have Been:
Single-school studies.
Nationally normed standardized tests in
economics/economic education (TUCE, TEU, TEL, TEK,
BET)
10
Important “Special/Irregular” Data Sources Have Been:
“The American Economy” television series and surveys (early 1960s)
Other educational films, audio-visual, and technology programs (late 1970s and
1980s)
The Pew Foundation program (late 1980s)
Advanced Placement (AP) exams in economics (1990s and continuing)
Major Field Tests (MFT) Also published by ETS. One study in 1999 by Walstad and
Allgood.
National Assessment of Educational Progress (NAEP) in economics 2006 and 2012
Other federal datasets U.S. National Center for Educational Statistics (NCES),
including periodic High School Transcript Studies. Baccalaureate and Beyond.
National Survey of Postsecondary Faculty (NSOPF). Release schedules vary, some
regular others irregular.
11
Important “Special/Irregular” Data Sources -- Continued:
Statewide assessments Some in early 1980s, but most after “No Child Left Behind”
reforms, and legislation and testing mandates of the early 1990s.
Surveys A few national surveys conducted by leading polling organizations such as
NORC and Gallup, 1990s. Others commissioned or conducted directly by economists
or economic educators on a state or occasionally national basis. Examples from all
decades WWII.
Studies of long term effects 2003 survey of college graduates from four public
universities (Florida Atlantic, Nebraska, North Carolina, Purdue) who took class in
1976, 1986 or 1996.
Multiple-Institution Database for Investigating Engineering Longitudinal
Development (MIDFIELD) Funded by NSF. First economic education studies in
2011 and 2012.
12
II. TRENDS AND ENDURING PATTERNS
IN ECONOMIC EDUCATION RESEARCH
B. Advances in Econometric and Statistical Methods
To help keep research in economic education on track (at least
with shorter lags) research conferences and training programs
and materials have been sponsored by such groups as the Joint
(later National) Council on Economic Education, the Journal of
Economic Education, and the American Economic Association
Committee on Economic Education.
Early conferences were tied to providing publication
opportunities for studies in the field, and to launching the JEE
in 1969. Later programs have been directed more at providing
training on new econometric methods.
13
Most extensive and expensive training programs at
Princeton University, 1987 and 1988. Emphasis on discrete
choice modeling and estimation of educational outcomes,
using William Greene’s LIMDEP software and the large
national data set from U.S. secondary students and
teachers, described earlier.
14
In 2010-11, the AEA Committee on Economic Education and the
Council on Economic Education sponsored the development of
four online modules providing training and practice in the use of
advanced econometric methods. Each module includes a “real”
data set and sample programs using the data in three different
statistical software packages: Stata, Limdep, and SASS. The
modules are available on line on the Web page of the AEA
Committee on Economic Education at
http://www.aeaweb.org/home/committees/AEACEE/Econometrics_
Handbook/index.php.
15
Module 1: Data Management and Heteroskedasticity Issues
Module 2: Endogenous Regressors with Natural
Experiments, Instrumental Variables, and Two-stage
Estimators
Module 3: Panel Data
Module 4: Sample Selection Issues
16
III. THE (SELECTED PAST AS PROLOGUE: SAY
SOMETHING EMPIRICAL WHERE SIZE (NOT
JUST STATISTICAL SIGNIFICANCE) MATTERS
In a more extensive review of research on precollege economic education
(Watts 2005), the key findings I stressed were that elementary and
secondary students can and do learn economics. They learn more when
teachers have more training in economics/economic education, and
when they use good instructional materials.
College students learn economics too, of course, although we have less to
say about faculty characteristics and instructional materials and
method that lead to more learning at this level – largely because there
is much less variance in instructors’ training in economics and in the
teaching methods they use.
17
I do not intend to minimize the importance of those
findings, but…
These general results are neither surprising nor compelling
enough to confirm current practice or to change the status
quo.
Because they don’t really change people’s views they are
more easily ignored, unless you believed economic education
was important before you knew about the research.
To some extent, then, the research becomes a form of
preaching to the choir.
18
Examples of an alternative way of framing and reporting
findings from past and (I hope) future research follow.
Not as many as I would like to have, nor are they as high a
share of all economic education research as I would like to
see.
Findings are drawn from earlier research – some from
very early work – I suspect many economic educators will
not be familiar with at least some of the results, at least
framed this way.
Frankly, I fear I may well be missing other good examples
of these kinds of results myself, simply because of the
framing issues and not being able to break through past
views and habits. Therefore, many examples come from my
earlier work, because I am familiar with the results.
19
1) Bach and Saunders (1965) -- secondary teachers with more college
economics coursework scored higher on a standardized test of basic
economics, but effects were only significant if teachers had taken at
least three courses, and not meaningfully sizeable unless they had
taken five or more courses (which raised mean scores by about seven
percent, after controlling for watching “The American Economy” TV
series, self reported class standing, and other personal and professional
characteristics).
A threshold effect at about four undergraduate courses on test scores
collected years after graduation has resurfaced in some later studies,
sometimes in a different context – for example, Lynch (1990) found that
secondary teachers who had taken four or more course increased
student scores on the TEL, while teachers with less economics
coursework did not.
The finding that taking only one or two college courses in economics
makes little long-term difference in understanding is also consistent
with the still provocative Stigler hypothesis.
20
Later research on the lasting effects of college principles courses using the
TUCE or other measures of knowledge pitched at a higher level than the
TEU or TEL showed positive gains from taking college principles courses
five years after graduation, in the range of about half the measured gains
found when students were tested as they completed the principles courses
(Saunders 1980). That represents a partial refutation of the Stigler
hypothesis, but Saunders also found evidence that the lasting gains were
concentrated more on application questions than on definitional, “recall
and understanding” questions, which generally supports Stigler’s criticism
of “encyclopedic” principles courses and textbooks that stress breadth and
rote learning rather than depth and critical thinking based on economic
analysis.
A final point on the observed threshold effect for economic understanding
at four or five courses is that this level of coursework basically translates
to the group of college students who decide to major or minor in
economics. Once again that raises questions about how much selfselection before graduation and self-interest after graduation drives these
outcomes, rather than the course and discipline content per se.
21
2) In four national surveys of college and university economics instructors
collected every five years, teaching methods in four different types of
undergraduate courses (principles, intermediate theory, statistics and
econometrics, and other upper-level field courses) were found to be
remarkably consistent and persistent, even across different Carnegie
classifications of schools. Using a non-linear Likert scale with five
responses, median responses showed over 80% of instructors lecturing
and writing on chalkboards (“chalk and talk”) in every class with little
or no use of other teaching methods more popular in other disciplines,
including cooperative and student-centered active learning methods.
Textbooks were required by over 80% of instructors in all types of
courses. Calculus was not seen as important by a majority of
respondents in any of the four types of courses, although the perceived
importance seems to be rising slowly over time, particularly in
intermediate theory and statistics and econometrics courses. Graphs
are viewed as extremely important in introductory and intermediate
theory courses and as somewhat important in the other two types of
courses – despite some evidence that graphs do not help students learn
more economics.
22
This summary picture of economics classes matches closely with what
students who took economics courses in the mid-1970s, 1980s, and
1990s say they remember about the courses. I.e, the economics courses
featured more straight lecture and graphs, with less discussion or use of
other teaching methods than other courses.
The survey results on basic and narrow teaching methods is one of the
exceptional items that has been widely cited by researchers in economic
education, perhaps illustrating the attraction of reporting direct
empirical “size” results.
23
3) Harter, Becker, and Watts (2011) report that national samples of U.S.
economists surveyed in 1995, 2000, and 2005 reported spending a little
more than half their time teaching, a little over 20% of their time doing
research, about 9% doing service, and presumably the rest of their time
on administration, consulting, and other activities. This is very much
in line with the NSOPF surveys of faculty members from all disciplines.
The economists also reported that reward structures in their
departments put more weight on research than their time allocations
reflected. Male faculty were more likely to devote slightly more time to
research than female faculty – again consistent with NSOPF data for all
disciplines – and to be employed at schools that put higher premiums on
research.
24
4) In the general education literature we see that different kinds of
teaching assessments – e.g., current student assessments, assessments
by former students, faculty self-assessments, peer assessments, and
assessments by external reviewers from outside a faculty member’s own
discipline – are only weakly correlated. Differences in one measure
explain half or often considerably less of the variation in other kinds of
assessments.
For a large national sample of U.S. economists, Bosshardt and Watts
(2001) confirmed that correlations between student and faculty self
assessments of teaching are low, and on most individual assessment
items very low (below 0.35).
Despite that, in surveys of economics department chairs at U.S. colleges
and universities, Becker and Watts (1999) and Becker, Bosshardt, and
Watts (2012) find that student evaluations of teaching are universally
and almost exclusively used as measures of teaching effectiveness.
Virtually no change in these practices across the 13 years between the
two surveys of department Chairs, except for greater use of online
collections of student evaluations.
25
The key question this all raises is whether schools and departments rely
exclusively on SET measures because they believe these are the best,
most important, or most reliable measures, or because they are
inexpensive to collect, or because they provide the appearance of caring
about students and teaching while allowing departments and schools
enough leeway to interpret results to reinforce assessments based
mainly on research or other performance measures.
26
5) In three papers dealing with economics learning and teaching at the
elementary, middle school, secondary, and undergraduate levels
(Bosshardt and Watts 1990 and 1994, and Watts and Bosshardt 1991),
at the precollege level student characteristics (including general
measures of ability or academic performance, gender, etc.) explained
about three to five times as much of the variance in student scores on
standardized tests as differences in instructor and school
characteristics. Across a sample of micro, macro, and “hybrid”
micro/macro principles classes at Purdue University, taught by doctoral
students (except in the hybrid class, where the doctoral students led
recitation sections once a week after students attended large lectures
taught by a faculty member twice a week), variance in measures of
instructor effects were about 46% as large as measures of student
variance in the macro classes, about 11% as large in the micro class, and
only 5% as large in the hybrid class. Consistently, across all of these
grade levels and course formats, student differences appear to be more
important than instructor or school differences; but the instructor
effects are large enough to have important effects on student
performance.
27
6)
Schaur, Watts, and Becker (2012) investigated school, department, and
faculty variables associated with instructors’ choices about what kinds
of assessment methods to use in different kinds of undergraduate
economics courses. Higher faculty teaching loads and larger classes
drive out the use of essay questions, term papers (except in principles
classes, where term papers are not seen as appropriate by most
instructors) and other kinds of writing assignments.
28
For example, as class size increases from 40 to 100 students in
principles classes, the probability of an instructor using multiple choice
questions rises from 76 to 88 percent and the probability of using essay
questions falls from 54 to 42 percent.
In upper-level field courses, as class size increases from 25 to 40 to 75
students, the probability of using term papers fell from 67 to 61 and
then 47 percent; in econometrics and statistics classes the probabilities
fell from 39 to 31 and 16 percent. Decreases in the use of essay
questions for the same increases in class size were: in intermediate
theory courses, from 77 to 71 to 56 percent; in other upper-division field
courses, 84, 80, and 71 percent; and in econometrics and statistics
courses, 47, 39, and 24 percent. The rise in the use of multiple choice
questions for the same class size increases were: in intermediate theory
courses, 34, 41, and 54 percent; in other upper-division field courses, 31,
47, and 80 percent; and in econometrics and statistics classes, 12, 21,
and 47 percent.
29
7) In the long-term effects transcript and survey study described in the earlier
section on databases, Allgood et al. (2004, 2010, and 2011) found that, compared to
a random sample of non-business majors, economics majors had a 17% higher
chance of earning more than $40,000 a year and a 14% greater chance of earning
more than $100,000 a year. (The survey data using these values was collected in
2003.) They also had an 8% greater chance of having been self-employed at some
point in their careers, and were 20% more likely to be paid at least partly on a
commission/bonus basis.
Economics majors were 4% more likely to own a home – probably because of higher
income. But even after controlling for income those who took some economics
coursework as undergraduates were less likely to have home equity of less than
$50,000.
They also saved 57% more, were more likely to invest in stocks, held about a third
fewer credit cards, and were more likely to pay the full balance on credit cards
each month. The economics majors were 8% less likely than the general majors
sample to have completed an advanced degree.
Differences between economics majors and business majors were generally not
significant, except that business majors were about 7% more likely than economics
majors to have been laid off at least once and were even less likely to have
completed a graduate degree.
30
8) In a survey of a representative sample of U.S. adults, Blinder and
Krueger (2004) found that over 60 percent of respondents reported they
“regularly or often” got information about economic policies from
television, with 49% checking newspapers, 35% checking both “friends
and relatives” and “political leaders,” and only 17% checking
“economists.” They also found that “ideology seems to play a stronger
role in shaping opinion on economic policy issues than either selfinterest or knowledge,” and noted that was “not terribly different” (p.
386) from conclusions offered by Fuchs, Krueger, and Poterba (1998) in
a survey of professional economists.
31
Using survey data from secondary social studies and economics
teachers, economists, and journalists, and comparing responses across
the groups using rank-order correlations, Becker, Walstad, and Watts
(1994) found that high school economics teachers held ideas closer to
responses by economists than other social studies teachers, but even the
economics teachers’ responses were closer to those of journalists than
those from economists. The relationship between more economics
coursework and reporting similar responses to economists on questions
about public policies was supported for broader groups of college
graduates in Allgood et al. (2010). Still, the key role and influence of
information sources about policy issues raises serious questions about
limits on the effects of economic education and coursework in this area.
32
IV. CONCLUSION
Despite the space setting out the importance of basic questions, data
sets, and advances in econometric and statistical techniques, the call for
more stress on developing and reporting results that have practically
important quantitative results is the main argument here in what we do
and how we do it.
Perhaps other economic educators will think it is not a major change, or
see it as a mistake to move in that direction. If so, still might be a
useful conversation to have.
The key point is that we should do something, or at least do more of
what we do well, to get more economists, educators, and policy makers
interested in and using research findings from economic education.
33
Many of the factors affecting answers to the bread and butter research
questions in economic education are quite general. Others are more
limited, in some cases affecting only the supply or demand for economics
coursework and majors.
Any factors that can change the supply or demand for economic
education represent a source of demand for new and updated studies in
the field.
To paraphrase Keynes, then, there are no settled policy conclusions in
economics or in economic education research because they are both
inherently contextual – tied to current problems and institutional
arrangements as well as to changes in production methods and
consumer preferences. As these “background” features change so too
may some of the answers to the enduring questions addressed by
researchers in the fields.
34
Every new generation will want some education and training in
economics given their roles as consumers, producers, and citizens, while
exactly what that education and training entails, and when and where
it is best delivered, will be changing rather than static. To anticipate
and recognize those changes, and to assess the effectiveness of
responses to them, will require new generations of research (and
researchers) in economic education.
35
Economic education may remain at relatively small and arguable
underrepresented field in both economics and education departments,
but I believe if we can attract some of the best and brightest economists
and educators to join in that research.
36
APPENDIX
Examples of factors that have changed answers to some of the basic
questions include:
1) the rapid rise of acceptance of experimental economics as an
alternative empirical methodology, and of behavioral economics as a
systematically different way to frame and structure a wide range of
research questions – especially those involving survey responses from
the general public.
37
2) greater access and reliance by students, businesses, and almost
everyone else on the Internet and other new information,
communications, and entertainment technologies; or
38
3) A wide range of recent public policies initiatives, many directed at
education reforms. For example:
Over the past decade U.S. political and education institutions
have devoted uncounted staff time and other resources writing
out performance measurement standards to promote (or at least
provide the appearance of promoting) accountability across all
levels of education.
Other programs and budget rules or guidelines have been
established to enhance affordability and “on time” graduation
rates for college students.
39