 To find if there are biases in my test beyond the
content domain itself (The structure of the U.S. federal
government) that may prejudice the test against those
who know english as a second language.
 Secondary- to find systematic flaws with the test that
effected the entirety of the testing group
 The test was given on a computer, half were given with
me present and the other half were administer online
with the promise that the test takers wouldn’t cheat.
The test was accessible through the internet.
 http://mrchalmershistory.wikispaces.com/testing
 There was a questionnaire before and after the test.
The first asked questions about the test taker and the
second asked questions about the test. Addition
feedback was often sought.
 The test was composed of 25 questions relating to the
structure of the U.S. government
 The first 9 questions were taken from the test bank of
the U.S. citizenship test.
 The remaining 16 were developed by me from other
sources
T
e
s
t
1
9
Test 10-18
 My findings in no way correlated to the studies about
the citizenship test on questions one through 9. Why?
 There was a large difference between the English as a
second language group and native speakers.
 Bias likely because of lack of knowledge of the content
domain
 There were flawed questions though but they were flawed
throughout all test groups
 7. What are two Cabinet level positions?
 Too vague- only two got it wrong
 15. How many cabinet positions are there?
 There is really no clear answer 16 currently
 19. Name two kinds of congressional committees.
 Again too vague Standing committees, select committees, Joint committees, conference
committees
 20. What is the political system- ideology that the United States
government follows
 One person said this test was ideologically biased
 Others had problems
 Possibilities for better phasing- 4,17,18
 Could have helped some with other questions 25-1, 9-8
 U.S. Citizenship and Immigration Services (2008)
Naturalization Test and Redesign Project: Civics Item
Selection Analysis (Form M-693)
 Turner, M. J., & Lake, S. (1989). U.S. government: A resource
book for secondary schools. Social studies resources for
secondary school librarians, teachers, and students. Santa
Barbara, Calif: ABC-CLIO.
 Burns, J. M. (2002). Government by the people. Upper
Saddle River, N.J.: Prentice Hall.
 Fulcher, G. (1997). Text difficulty and accessibility: Reading
formulae and expert judgment. System 25 4, pp. 497–513