Dr. Olson’s Comments on Students’ Responses to Forum 9
Forum 9: Test Prep
After reading Mehrens & Kaminski, Popham, and the two articles on the Internet (Study and Test Prep
Strategies, and No Pain, High Gain), consider the test preparation strategies you have seen employed in
the schools. What is your reaction? Do you think the procedures used in the schools are ethical, or
educationally defensible? As a school leader, what would you suggest your teachers do to prepare
students for the EoG or EoC tests? In framing your response, refer to material in all four articles.
Keeley, you wrote, “I know the state’s DPI website has released sample test items that my teachers
utilize to help students practice the format and test-taking strategies in the weeks leading up to the
EoGs/EoCs. Although I do understand the argument of possible outsiders’ thoughts of using such
releases could be unethical, teachers are not stealing or utilizing the actual assessment used for the final
test.” Actually, I think Popham would say that the use of those items in the way you describe is not
exactly unethical; rather, not educationally defensible. My own take is that it is ok to use the items to a
limited extent. Popham’s argument is that education should be aimed at developing knowledge and
skills beyond what is needed to answer multiple-choice test items. Surely we are not educating our
students to perform well on Jeopardy, or to be “smarter than a 5th grader. The test prep strategies you
describe in your second paragraph are the more appropriate strategies…and, they do not take all that
much time. Also, I agree with you that using formative assessment, in which test-taking strategies can be
taught at the same time, is a good way to go.
Pam, thanks for the Gipps (1994) quote, “It is not that teachers want to narrow their teaching, nor to
limit unduly students’ educational experience, but if the test scores have significant effects on people’s
lives, then teachers see it as part of their professional duty to make sure that their pupils have the best
possible chance they can to pass the test.” As I hope you, and everyone else is aware, this course is
aimed at the principles and ideas embodied in Popham’s and Mehrens and Kaminski’s test prep
approach. In the Craft an Assessment Exercise, you first had to identify well-articulated learning target,
then you had to construct tests to assess students’ performance relative to those targets. The next
obvious step is to teach for mastery of those targets. Your child’s teacher apparently has a good grasp
on how to do Test Prep. And Pam, in your reply to yourself, you support and extend what your child’s
teacher provided. Good list. I hope everyone else reads it. Also, Amy, in your reply to Pam, I like your
suggestion about taking the Test Prep strategies slowly. However, I think one per week would suffice.
Tracee, I agree that “getting copies of previous EOC’s and letting students know questions which are
addressed on the EOG or EOC after you have seen the initial evaluation,” definitely is on the boarder of
being unethical; it is clearly not educationally defensible, for two reasons. First by focusing on what was
on previous tests, instruction can easily miss important knowledge and skills that might appear in items
on a new (or alternate form of the) test. Second, if students performances improve, is it valid to infer
that they have gained the knowledge and skills in the SCOS? That teachers acquire knowledge of what
appears on the state tests. either through exposure to the tests or by conversing with other teachers
who have seen the tests, is well known. It is one of the factors contributing to improvements in test
scores over years. As Amy implies, in her replay, when they have specific information about what will be
tested, teachers cannot help but prepare their students for it. To me this seems to be human nature.
The problem lies with test security.
Olivia, I would hope that all teachers in NC have read the Testing Code of Ethics. Failure to do so could
lead to infractions that are taken quite seriously. You provided some excellent examples of how certain
test-prep strategies might lead to increased test scores without concomitant increases in content
mastery. In agreement with Stefanie’s reply to your post, and Elizabeth’s post following yours, I would
say that, If teachers can teach students, in the few days before the tests, to perform better on the tests,
why can’t they do this throughout the year? In regards to your comments about released items and test
format, see my earlier comments to Tracee’s post, earlier.
Elizabeth, You are correct, when the instructional intention is to “teach English, … students [should] be
reading literature and not just excerpts from a test prep book.” This, obviously is where the importance
of clearly articulated learning target come into play. However, teaching vocabulary is an “enabling,” but
not a critical, learning target. Vocabulary acquisition is probably better assesses in the process of
Assessment for Learning. Also, I agree with you that when your students, “learn to write and formulate
their own arguments” you are not violating ethical standards. However, you could generate some
original prompts. As for those teachers who copy tests, that, it seems to me, is clearly an infraction of
the Testing Code of Ethics. As you stated, “it’s important that the leadership of the school keep the
teachers from going to unethical testing practices.”
In replying to your post, Elizabeth, Kendra wondered about your school’s use of previous tests to help
students understand what students have learned. She wondered if this was unethical. It my opinion, it
depends. If the tests that are held are high-stakes tests that will be used again, then, yes, it would be
considered unethical. On the other hand, if those tests are locally-constructed tests, and not used for
grading students, but rather for identifying students’’ strengths and weaknesses, than I would say OK.
That would be a formative use of the tests.
I can agree with you, Misty, about the use of ClassScape. It can be very useful in formative assessment. I
believe, as you do, that the “drill and practice” should not be over done. I believe, also, that students
should be exposed to test items in formats other than multiple choice.
Brian, You are not alone in wanting high test scores for your students. But how you get them there is
important. Surely you want to be able say that your students performed well on the tests because they
did a good job at mastering the content in the SCOS. As you said yourself, when educationally
indefensible test-prep practices are employed, “… data reflected by their scores may not be an accurate
account of the content domain tested.” It seems to me that teachers, if they really think about their
mission, would try very hard to be consistent with Popham’s defensibility criteria. After all, isn’t the
primary purpose of teaching to imbue our students with credible knowledge and skills? And, isn’t it the
administrator’s job to see to it that teachers are doing this? And, Amy, Olivia, Stephanie, Jessica, and
Kendra, thank you for your great (and honest) replies to Brian’s post.
Stephanie, Your post was wonderful. I hope everyone read it. You had so many good things to say that I
cannot possibly recount them here. You give some really good examples of the insidiousness of
questionable, if not indefensible, test-prep practices find their way into the schools. But, Amy, in your
reply, while mnemonics are helpful for memorizing facts, just remember, that your is to teach more than
fact…it is to teach students to think and be able to use facts for higher-level accomplishments.
In reply to several of the replies to your post, Stephanie,I have two comments to make. First, teaching
the test-prep strategies suggested earlier by Pam and some of the others really do not need to consume
all that much classroom time. Second, the rest of the classroom time is better spent teaching the
content of the SCOS. Again, as school leaders, one of your responsibilities is to see that teachers spend
most of their time teaching. Kevin, in his post following yours, apparently agrees.
I am somewhat appalled, Kendra, that teachers at your schools “are currently preparing booklets [from
previous year testing materials…and…from websites that provide examples of test taking items] for
students to prepare them for the end of the year testing.” Surely the teachers’ time could be better
spent helping students master their learning targets. Furthermore, your schools’ use of boot camps to,
“for four weeks instruction [only] in the areas that are being tested,” is, in my opinion, and with which
you appear to agree, also reprehensible.
Amy, a long post, but a good one! You’ve given your classmates a lot to ponder. I liked your suggestions
such as the one for giving students a classroom test, “let’s stop a moment and see what we have learned
and understand in our simple addition unit of study.” Also, maybe the two teachers whose students
performed better on the tests were spending more time covering the SCOS learning targets. You are on
target, though, in regards to easing students’ anxiety. The literature, I believe, gives many suggestions
for doing so. Just Google “test anxiety.”
Also, Amy, with respect to schools’ and teachers’ use of benchmark tests, we’ve discussed them before,
in Forum 2. Locally-constructed benchmark tests often possess the same degree reliability and validity
as high-stakes state tests or commercially-available tests. This could be problematical. For this and other
reasons, it is also questionable whether benchmark tests should be used for summative proposes such
as giving report-card grades. In my opinion, benchmark tests are best used for formative assessment.
Katherine, Another long post with a lot of good information and material. Thanks for pointing out that
letter grades often do not provide enough information to be useful. Perhaps when we discuss studentled conferencing that will help. Your paragraph, where you discuss Whitehead, is worth a read by
everyone. I especiall liked your final statement, “Using assessment methods throughout the school year
that focuses on a small number to learning targets, fosters effective communication regarding students’
learning needs, and improves teachers’ instructions will serve as a successful framework for EOG or EOC
tests.”
Nikkie, I’d like to comment on your statement, “I know that the article says these methods are not
defensible and unethical. However, I disagree. I feel students need practice with how test questions are
stated to allow them to be comfortable with the format. The format of the question does not need to
keep students from being able to show their mastery of the content.” The authors of the articles
assigned as readings are quite clear in pointing out that “learning” the content goes well beyond being
able to answer, correctly, multiple-choice questions. In earlier readings, it was pointed out that such
questions are limited in terms of what they are capable of assessing. Regardless of what you might
think, your job as a teacher is not to teach kids to take (or even do well) on an end of course or end or
year test; it is to TEACH students the content in the SCOS. If using those strategies that you seem to
agree with, but others call indefensible or unethical, results in higher test scores, than what do you infer
when students do achieve higher test scores? Will you take credit for having taught them the content?
Regina, I think that what you suggest (or what you would do) in your post is great. Your post is well
worth reading by everyone. It provides good guidance for school administrators. Your Abbot and
Costello reference was great.
Finally, thank you Jessica, for sharing your experiences at the Asheville conferences, and for the links.
Everyone in this course need to read your post and visit the links. To borrow a line from Dillon, “The
times, they are a changing.” It is going to be interesting to see how all this unfolds.