The Test of Economic Literacy
Content, Development,Uses
William Walstad, Professor of Economics
University of Nebraska-Lincoln
Webinar: November 13, 2013
Objectives for Webinar

Describe the TEL content

Explain the TEL development

Present the TEL national results

Suggest ways to use TEL

Help you obtain a TEL copy


Examiner’s Manual
Test forms
TEL Content

For high school (grades 10-12)

Multiple-choice test



Classroom test



45 items on each form
two parallel forms (A and B)
Basic economics
AP/honors economics
Based on CEE content standards
TABLE 1: Distribution of TEL Items by Economics Standards
Form A
Standard with selected key concepts
Form B
Items
Total
Items
Total
1, 2, 3
3
1, 2, 3
3
4
1
4
1
3. Economic systems and allocation
5, 6
2
5, 6
2
4. Economic incentives—prices, wage, profits,
7, 8
2
7, 8
2
5. Voluntary exchange and trade
9, 10
2
9, 10
2
6. Specialization and comparative advantage
11, 12
2
11, 12
2
7. Markets and prices
13, 14
2
13, 14
2
8. Supply and demand
15, 16, 17
3
15, 16, 17
3
9. Competition
18, 19, 20
3
18, 19, 20
3
21, 22
2
21, 22
2
23, 24, 25
3
23, 24, 25
3
12. Interest rates
26, 27
2
26, 27
2
13. Labor markets and income
28, 29
2
28, 29
2
30
1
30
1
15. Physical and human capital investment
31, 32
2
31, 32
2
16. Economic role of government
33, 34
2
33, 34
2
17. Government failure, special interest groups
35
1
35
1
18. Output, income, employment, and price lev.
36, 37, 38,
39, 40
5
36, 37,
38, 39, 40
5
19. Unemployment and inflation
41, 42
2
41, 42
2
20. Fiscal and monetary policy
43, 44, 45
3
43, 44, 45
3
1. Scarcity, choice, productive resources
2. Decision-making, marginal analysis
10. Economic institutions
11. Money and inflation
14. Entrepreneurship
Total Number of Questions
45
45
Item & Rationale
ITEM
1. The opportunity cost of a new city park is
the
RATIONALE
The opportunity cost of producing a good or
service is the next best alternative good or
service that might have been produced with the
A. cost of staff and maintenance for the
same resources. In other words, opportunity
park.
B. increased congestion from traffic around cost refers to what is forgone once money or
resources are used for a specific purpose.
the park.
C. best alternative use of resources given [1/4/5]
up for the park.
[Code for bracket item: Standard / Grade
D. lack of personal incentive for people to
Level / Benchmark (CEE, 2010)]
take care of a public park.
Parallel Items (A & B)
1. The opportunity cost of a
The opportunity cost of a new
new public high school is the
city park is the
A. money cost of hiring
teachers for the new school.
B. cost of constructing the new
school at a later date.
C. change in the annual tax rate
to pay for the new school.
D. other goods and services
that must be given up for
the new school.
A. cost of staff and
maintenance for the park.
B. increased congestion from
traffic around the park.
C. best alternative use of
resources given up for
the park.
D. lack of personal incentive
for people to take care of a
public park.
Cognitive Levels
 Knowledge
(recognition and recall)
 Comprehension
(understand the meaning)
 Application
(apply concepts or doing analysis)
Cognitive Level
Distribution
 Knowledge……….
13%
 Comprehension…… 31%
 Application………
56%
Table 1: Content & Cognitive Levels
Standard with selected key concepts
Form A
Know Comp
1. Scarcity, choice, productive resources
2, 3
2. Decision-making, marginal analysis
3. Economic systems and allocation mechanisms
4. Economic incentives—prices, wage, profits, etc.
App
Total
1
3
4
1
5, 6
7
2
8
2
5. Voluntary exchange and trade
9, 10
2
6. Specialization and comparative advantage
11, 12
2
7. Markets and prices
13, 14
15, 16,
17
18, 19
2
8. Supply and demand
9. Competition
10. Economic institutions
11. Money and inflation
12. Interest rates
13. Labor markets and income
14. Entrepreneurship
22
25
20
21
23
28
30
24
26, 27
29
3
3
2
3
2
2
1
15. Physical and human capital investment
32
31
2
16. Economic role of government
33
34
2
35
1
40
5
2
3
45
100.0
17. Government failure, special interest groups
18. Output, income, employment, & price level
19. Unemployment and inflation
20. Fiscal and monetary policy
Total Number of Questions……………..........
Percent of Total……………………………
36, 37 38, 39
41
44
6
14
42
43, 45
25
13.3
31.1
55.6
Test Development:
Preparation
 review and rate existing items
(good base– 3 editions)
 develop new test items
 conduct a field test
 prepare norming version
 Timeline (2011-2012)
 Spring-Summer 2011
 Prepare new test questions
 Fall 2011
 Field test
 Prepare final norming version
 Administer norming version
 Winter 2012
 Administer norming version
 Spring 2012
 Administer norming version-posttest
 Summer 2012
 Data analysis and write manual
Test Results

Overall statistics

Total (with and without econ)


Basic (with and without econ)
Advanced (with and without econ)

Percentile norms

Item difficulty & discrimination

Construct validity data
Aggregate Statistics for TEL Norming Sample
Form A Form B
Sample Size
Number of Students
3,682
3,686
Percent with Economics
50
49
Reliability
Coefficient alpha
Means
.91
.90
Overall
23.32
23.17
[A = 3,682; B = 3,686]
With Economics
[A = 1,829; B = 1,816]
Without Economics
[A = 1,853; B = 1,870]
(9.70)
27.03
(9.77)
19.09
(8.12)
(9.29)
27.03
(9.30)
19.43
(7.60)
Notes: (1) Sample sizes are in brackets.
(2) Standard deviations are in parentheses.
(3) Also data for basic and advanced
Percentile Norms
With Economics
Without Economics
Raw
Score
Basic
Advanced
Basic
Advanced
(n = 1,494) (n = 335) (n= 1,702) (n = 151)
45
44
43
42
41
99
98
99
98
92
85
79
99
98
40
39
38
37
36
97
95
93
90
87
71
64
59
54
47
99
98
97
97
97
96
94
92
35
34
33
32
31
84
81
77
73
70
43
39
35
33
27
97
96
94
93
92
87
83
78
76
72
30
67
25
90
68
Item Discrimination and Percentage
of Correct Responses
Corrected
Item–Total
Correct Correlation
Answer (n = 3,682)
Item
Percent Correct
With
Economics
(n = 1,829)
Without
Economics
(n = 1,853)
1
2
3
D
B
C
.46
.46
.47
61.8
57.4
72.0
34.5
32.5
49.0
4
5†
6
7†
8
9
10
D
B
C
A
C
A
C
.40
.37
.45
.47
.39
.31
.43
50.0
55.6
77.6
70.4
54.2
55.2
70.0
36.9
39.7
50.2
44.6
42.3
45.8
57.4
Percentage Response to
Each Alternative
Item
1
2
3
4
5†
6
7†
8
9
10
A
15
18
8
8
8
12
57*
14
50*
10
B
13
45*
12
12
48*
10
8
17
39
15
C
20
20
60*
34
17
64*
18
48*
5
64*
D Blank
48*
4
13
5
15
5
43*
4
20
7
10
4
12
4
18
4
4
3
7
5
Test Validity
 Content validity:
Based on Voluntary National Content
Standards in Economics (CEE, 2010)
 Construct validity:
Expected differences with and without
economics
Descriptive Statistics
(with and without econ)

Gender

Grade level

Race/ethnicity

Verbal ability

Communication skills

School size

Student/teacher ratio

Percent free lunch

Type of community

Region
Descriptive Statistics Within the Total Norming Sample: TEL Form A
With Economics
Without Economics
Mean Std. Dev. Number
Mean Std. Dev. Number
By gender
Female
26.17
9.43
893
18.89
7.70
942
Male
27.82
10.02
936
20.51
8.46
911
By grade level
Grade 9
Grade 10
Grade 11
25.65
24.26
25.90
9.62
10.00
10.03
178
87
391
18.46
17.92
19.20
7.86
7.27
7.67
194
241
475
Grade 12
27.80
9.60
1,773
20.63
8.48
943
21.34
9.72
83
15.98
6.60
278
33.45
27.28
24.61
17.92
26.87
9.13
9.55
8.95
9.06
10.04
131
1,343
171
13
15
22.19
20.78
17.79
15.50
18.09
8.28
8.24
7.44
7.99
4.13
78
1,219
191
14
11
24.34
9.74
73
18.63
7.92
62
19.09
22.24
6.73
8.32
434
602
14.89
19.02
5.47
6.87
575
631
28.90
9.31
793
24.59
8.45
647
27.15
9.84
1,690
19.89
8.12
1,683
26.32
8.98
28
17.56
8.53
36
25.19
8.70
111
17.73
7.74
134
By race/origin
African
American/black
Asian
Caucasian/white
Hispanic
Native American
Pacific Islander
Other
By verbal ability level
Low
Middle
High
By communication
skills
Best in English
Best in another
language
Equal in English
and another
language
With Economics
Mean
Std. Number
Dev.
Without Economics
Mean
Std. Number
Dev.
26.54
9.59
939
19.51
7.98
1,054
28.18
9.68
824
20.26
8.46
688
By student/teacher
ratio
17 to 1 or less
More than 17 to 1
27.87
27.00
9.79
9.58
642
1,121
20.18
19.63
8.29
8.10
690
1,041
By % free lunch
17 or less
More than 17
28.45
24.71
9.55
9.14
976
616
21.60
18.46
8.12
7.81
591
950
By type of
community
City
Suburb
Town
Rural
27.70
27.12
26.71
26.75
8.85
10.55
8.83
9.17
371
638
311
413
20.67
19.82
18.51
19.38
8.34
8.11
7.75
8.15
544
487
305
436
By region
Northeast
Midwest
South
West
27.24
26.32
29.17
27.03
9.07
9.88
9.65
9.55
91
995
312
431
16.11
20.62
18.37
19.83
6.01
8.23
7.70
8.41
63
947
538
305
All students
27.02
9.77
1,829
19.68
8.12
1,853
By school size
Fewer than 1,500
students
1,500 or more
students
Uses of the TEL

Overall test

Pretest only (knowledge)

Posttest only (knowledge)

Posttest—Pretest (learning)

Comparative Norms—national data

Item analysis


Item rationales


Right and wrong answers
Explain why item is correct
Reports and evaluations
For more test information:
TEL Examiner’s manual
Thank You!