The Test Assessment Questionnaire:
A Tool for Student Self-Assessment
After the Midterm Exam
Katherine M. Sauer
Metropolitan State University of Denver
dr.katherine.sauer@gmail.com
William Mertens
University of Colorado at Boulder
Economics Teaching Conference November 8th – 9th 2012 Orlando, FL
Students are Overconfident
Walstad (2001) - suggests concepts from behavioral
economics (e.g. overconfidence) could be used to explain
student behavior
Falchikov and Boud (1989) - students have grade
expectations that are higher than the typical distribution
for the course
Grimes (2002) - has principles students predict exam
scores and finds overconfidence in their understanding of
material
Nowell and Alston (2007) - instructor grading practices
can influence the degree of overconfidence
What happens
when students are
overconfident?
Students may start to blame the
instructor for the disconnect
between (perceived) effort and
desired grade.
Students may label the instructor
as “unfair” or the course simply
“too hard”.
Grimes, Millea, and Woodruff (2004) - degree to which
students accept personal responsibility for performance
affects their evaluation of teaching effectiveness and
course satisfaction
Carrell and West (2010) - in course evaluations, students
reward professors who increase achievement in the
contemporaneous course, not those who facilitated deep
learning for subsequent courses
Millea and Grimes (2002) - instructors need not “waterdown” courses in order to receive favorable course
evaluations… they can positively influence evaluations by
addressing negative student attitudes about forthcoming
coursework
How can I help
my students
become
more self-aware
in their learning?
Get them to focus on the type of
mistake they are making instead
of the number of mistakes they
are making.
Get them thinking about the link
between their study activities and
their exam performance.
The Test Assessment Questionnaire
This instrument guides students through an
analysis of their midterm exam mistakes.
• questions about exam preparation activities
• studying activities for the course in general
• formulate a study strategy for the rest of the
course
The TAQ starts with a series of multiple choice questions
regarding course preparation activities.
How often did you miss our class?
a. never
b. 1 or 2 times
c. more than 2 times
For how many chapters did you complete the Reading
Outline?
a. all
b. most of them
c. a few of them
d. none
When did you complete the Reading Outlines?
a. before we covered the topic in class
b. after we covered the topic in class
For how many chapters
did you complete the
online Homework?
a. all
b. I missed 1 or 2
c. I missed 3 or more
How often did you come to
office hours or email the
professor for clarification
on the material?
a. never
b. 1 or 2 times
c. 3 or more times
When did you start working through the exam
information / review sheet?
a. as soon as it was available
b. 2 weeks before the exam
c. 1 week before the exam
d. the night before the exam
e. I didn’t work through it.
For how many chapters did you work some of
the Optional Problem Set questions?
a. all chapters and all problems
b. most chapters and problems
c. a few chapters and problems
d. none
How did you prepare for this exam?
How many hours did you spend preparing for the exam?
How does your exam score compare with your
homework average? Why do you think that is the case?
Go through the questions you answered incorrectly
on your exam.
For each question, choose the reason you feel you
got the answer wrong.
Write the number of each question in the table next
to the reason.
Do you notice any patterns with the type of
question you missed? Explain.
What letter grade do you hope to earn in this course?
To achieve your goal, what is your strategy for studying
between now and the next exam?
Logistics
 TAQ worksheet is given to students when the exam
is handed back
 due one week later and counts as a homework grade
 in the class period when it is due, debrief with study
strategies for each type of common mistake (10
minutes)
- students can come to office hours for additional
guidance
Study Strategies for Common Mistakes
Didn’t know a definition
- make flashcards (as we go along, not at the end!)
Couldn’t apply a definition I knew
- think of examples /illustrations for each concept
and add to flashcards
Didn’t read the question/answers carefully
- “mark up” exam as you go
- every 3 to 5 questions, physically set down pencil,
close eyes, take deep breath
Knew theStudy
answer
but couldn’t
come up with
it during the
Strategies
for Common
Mistakes
exam
- check out the test-taking strategies and testanxiety workshops on campus
Didn’t know how to set the problem up
- for each type of problem we work, go back and
write out in words the general steps we took to
solve the problem
Used the wrong formula
- copy formulas onto one piece of paper
- next to each, describe in words when you would
use the formula
- copy an example problem next to each formula
Study Strategies for Common Mistakes
Debated between two answers and choose the wrong one
- good news! able to eliminate 2 answers
- probably know material at 80-85% level
- missing some detail in that last 15-20%
- spend a little more time and effort on the details
Just didn’t know the material
- students usually know why
Other
- come to office hours and we’ll come up with some
specific strategies for you
Study Strategies for Common Mistakes
1) it guides students toward more selfawareness in their studies
Benefits of
Using the
TAQ
2) it can be used as a part of assessment
and assurance of learning efforts
3) may improve students’ final exam
scores over the midterm
Student Self-Awareness
On the course evaluations: Please comment on the
experience of completing the Test Assessment and/or
meeting with me to discuss it.
- It made me improve my studying skills.
-Good, it gave me one-on-one time with you to figure out
how I could improve.
-Helpful in learning how I make mistakes and how to fix
them better.
- It helped me to personally analyze my test taking and
study habits. Very helpful! I have never had a teacher
who cared enough to take the time to help me improve in
this aspect.
-Your tips helped me in other classes and got me running
again and I don’t feel as stressed. My mom thanks you.
There was no negative feedback on this question.
Assessment & Assurance of Learning
More and more time is devoted to these activities.
We assume that students are doing their part in the
learning process.
To what extent are they?
Routinely, I find students who earn a D or F on an exam
reveal that they simply didn’t study.
- provides context to “does not meet expectations”
Improve Students’ Final Exam Scores
Experiment:
Fall 2011 pilot – University of Colorado
All Students in Principles of Microeconomics
10 course sections, 1713 students
62 recitation sections
Design:
Course instructors do not modify their class time or
teaching activities in any way.
Each TA that teaches two or more recitation sections
has a control and experiment section assigned to them.
The experiment will take place during the recitation
class in which the midterm is handed back.
Control Sections: TAs spend the class time going over
the exam as they normally would and will offer some
general study strategies.
Experiment Sections: Students will be given the correct
answers and will spend the class time going over the
exam by themselves while completing the TAQ.
After the TAQs are completed, the TA discusses study
strategies that complement each of the common mistake
categories.
All students complete Duttweiler’s (1984) Internal
Control Index.
- many such instruments
- argue this one is most reliable and valid
one for adults
- normed on college students
- 28 questions, Likert-type scale
Half of the items are worded so that high internally
oriented respondents are expected to answer at the
"usually" end of the scale and the other half at the
"rarely" end.
The "rarely" response is scored as 5 points on items
1,2,8,11,14,17,19,22,23,24,26,27.
This produces a possible range of scores from 28 to
140 with higher scores reflecting higher internal locus
of control.
For college students:
MEAN 99.3- 120.8
Data:
- ICI scores
- scores for homework, recitation, exams
- gender, race, GPA, etc from Office of
Institutional Research
Two course instructors opted out of the experiment.
n = 1337
27 treatment recitations, n = 667
27 control recitations, n = 670
But not all sections were full.
We received 522 first-round ICI surveys from 28
different recitations.
We received 488 second-round ICI surveys also from 28
different recitations (but not exactly the same recitations
as those for the first-round surveys).
We received 234 TAQs from 13 different recitations.
Data from One Course Section (n = 188)
Treatment (n = 94)
27 students completed the ICI twice and the TAQ.
12 students completed one ICI and the TAQ.
Control (n=94)
35 students completed the ICI twice and the TAQ.
30 students completed one ICI and the TAQ.
Fall 2012 – Rerun the Experiment
2 sections of Principles of Microeconomics
Course instructors are actively supportive
TAQ and ICI will have weight in course grade
Questions? Comments?
Keep in touch:
dr.katherine.sauer@gmail.com
Katherine Sauer, Ph.D.
@yogiconomist
Katie Sauer Hart
303-556-3037