WESTERN UNIVERSITY
LONDON
CANADA
Department of Psychology
2014-2015
PSYCHOLOGY 3840F - Section 001
RESEARCH IN TEST CONSTRUCTION
1.0
CALENDAR DESCRIPTION
A review of methods for developing psychological tests, questionnaires, and surveys. Topics
will include reliability, validity, norming, standardization, and strategies of test construction.
Opportunities for developing measures of human abilities and personality will be provided.
Strongly recommended for students planning graduate training in psychology and related
disciplines. 3 lecture hours, 0.5 course.
Antirequisites: Psychology 2080A/B.
Antirequisites are courses that overlap sufficiently in content that only one can be taken for
credit. So if you take a course that is an antirequisite to a course previously taken, you will
lose credit for the earlier course, regardless of the grade achieved in the most recent course.
Prerequisites: Minimum grade of 60% in both Psychology 2800E and 2810, or minimum
grade of 70% in Psychology 2820E (or a minimum grade of 60% in both Psychology
2830A/B and 3830F/G at Huron).
Unless you have either the requisites for this course or written special permission from
your Dean to enroll in it, you may be removed from this course and it will be deleted from
your record. This decision may not be appealed. You will receive no adjustment to your
fees in the event that you are dropped from a course for failing to have the prerequisites.
2.0
COURSE INFORMATION
Instructor:
Lectures:
Office Hours:
Teaching Assistant:
Office Hours:
S. Paunonen (paunonen@uwo.ca; 519-661-2060)
Thursdays, 2:30-5:30, Room 3120 SSC
Mondays, 4:30-5:30, Room 6304 SSC
Rachel Plouffe (rplouffe@uwo.ca)
Wednesdays, 1:00-3:00, Room tba SSC
If you or someone you know is experiencing distress, there are several resources here at Western
to assist you. Please visit: http://www.uwo.ca/uwocom/mentalhealth/ for more information on
these resources and on mental health.
Please contact the course instructor if you require material in an alternate format or if you require
any other arrangements to make this course more accessible to you. You may also wish to
contact Services for Students with Disabilities (SSD) at 519-661-2111 (ext. 82147) for any
specific questions regarding an accommodation.
3.0
TEXTBOOK
Kline, P. (1986). A handbook of test construction. London: Methuen.
Paunonen, S. V. (2012). Psychology 3840 lecture notes. UWO Bookstore.
Psych/3840
4.0
COURSE OBJECTIVES
The primary objective of this course is to introduce the student to the major methods of
psychological test construction. Those methods will be examined in terms of their underlying
psychometric theory related to reliability and validity estimation, standardization, and equating.
Students will write and evaluate test items by constructing an in-class personality questionnaire.
5.0
EVALUATION
Students will be required to write one 2-hour midterm exam and one 3-hour final exam. Exam
items will be of the short answer, short essay, and problem-solving variety. The final exam will
be cumulative in its content coverage. The graded midterm along with a complete exam scoring
key can be examined with the TA. In addition to the two exams, a written project (2500 words)
on item analysis will be assigned. Details about that project will be available on a separate
handout.
The Psychology Department does not require instructors to adjust their course grades to targets,
but the expectation is that course marks will be distributed around the following averages:
70% 1000-level and 2000-level courses
72% 2100-level to 2990-level courses
75% 3000-level courses
80% 4000-level courses
The Psychology Department follows these University of Western Ontario grading guidelines (see
http://www.uwo.ca/univsec/pdf/academic_policies/general/grades_undergrad.pdf):
A+
A
B
C
D
F
6.0
90-100
80-89
70-79
60-69
50-59
below 50
EXAM SCHEDULE
Midterm
Project
Final
7.0
One could scarcely expect better from a student at this level
Superior work that is clearly above average
Good work, meeting all requirements, and eminently satisfactory
Competent work, meeting requirements
Fair work, minimally acceptable
Fail
DATE
WEIGHT
LOCATION
Oct. 16
Nov. 27
Dec. 6-17
.30
.30 (-5% p.d. late)
.40
EC 2155
6304 SSC
tba
LECTURE TOPICS AND ASSOCIATED READINGS
The lecture topics and the assigned readings are listed on a separate page. The textbook and all
readings can be found online. The Lecture Notes are sold through the Bookstore.
8.0
STATEMENT ON ACADEMIC OFFENCES
Students are responsible for understanding the nature and avoiding the occurrence of plagiarism
and other scholastic offenses. Plagiarism and cheating are considered very serious offenses
because they undermine the integrity of research and education. Actions constituting a scholastic
offense are described at the following link:
http://www.uwo.ca/univsec/pdf/academic_policies/appeals/appealsundergrad.pdf
Psych/3840
8.0
STATEMENT ON ACADEMIC OFFENCES (continued)
As of Sept. 1, 2009, the Department of Psychology will take the following steps to detect
scholastic offenses. (a) All multiple-choice tests and exams will be checked for similarities in
the pattern of responses using reliable software, and records will be made of student seating
locations in all tests and exams. (b) All written assignments will be submitted to TurnItIn, a
service designed to detect and deter plagiarism by comparing written material to over 5 billion
pages of content located on the Internet or in TurnItIn’s databases. (c) All papers submitted for
such checking will be included as source documents in the reference database for the purpose of
detecting plagiarism of papers subsequently submitted to the system. Use of the service is
subject to the licensing agreement, currently between The University of Western Ontario and
Turnitin.com (http://www.turnitin.com).
Possible penalties for a scholastic offense include failure of the assignment, failure of the course,
suspension from the University, and expulsion from the University.
9.0
OTHER INFORMATION
The information below pertains to make-up exams, electronic aids, online resources, etc.
MAKE-UP EXAMS:
1. Only under special circumstances will a student be allowed to write a make-up test or
exam or be given an extension on an assignment. These include medical or
compassionate reasons substantiated by the proper documentation (see below) approved
by your faculty’s Academic Counselling Office.
2. If the missed exam is the final examination (set by the Registrar’s Office) the student
must contact the Academic Counselling Office to request a permission form for writing a
Special Final Examination. That form must be signed by the instructor, the department
chairperson, and the Dean.
3. A student missing an exam or assignment must notify the instructor on the day of the
exam or deadline, or on the following day at the latest.
4. The student should try to notify the instructor in person or by telephone, rather than by
email. If leaving a phone mail message, be sure to leave your telephone number.
5. A student who misses an exam or deadline for other than medical or compassion
reasons, or who is unable to substantiate a claim, or who fails to notify the instructor by
the day following the exam or deadline will be assigned a grade of zero for that
evaluation. Late written assignments will be assessed a penalty of 5% per day.
6. Make-up exams or extensions to deadlines will not be scheduled to accommodate
students’ travel or work plans.
MEDICAL NOTES:
In May, 2008, The University of Western Ontario’s Senate approved a new medical
note policy, which affects all students. The following is an outline of that policy. For
more detailed information and forms, please visit:
https://studentservices.uwo.ca/secure/index.cfm
For further policy information please visit:
http://www.uwo.ca/univsec/handbook/appeals/accommodation_medical.pdf
Psych/3840
9.0
OTHER INFORMATION (continued)
MEDICAL NOTES: (cont.)
Notes from Family Physicians and Walk-In Clinics
A Western Student Medical Certificate (SMC) is required where a student is seeking
academic accommodation. This documentation should be obtained at the time of the
initial consultation with the physician or walk-in clinic. An SMC can be downloaded
under the Medical Documentation heading of the following website:
https://studentservices.uwo.ca/secure/index.cfm.
The completed SMC should be returned to the student’s home Faculty Academic
Counselling Service.
Notes from Student Health Services
Students must request an SMC be filled out by the attending physician. There is a charge
for this service. The completed SMC should be returned to the student’s home Faculty
Academic Counselling Service.
Notes from Hospital Urgent Care Centres or Emergency Departments
Students should request that an SMC be filled out. Students may bring this form with
them, or request alternative Emergency Department documentation. Documentation
should be secured at the time of the initial visit to the Emergency Department. Where
it is not possible for a student to have an SMC completed by the attending physician,
the student must request documentation sufficient to demonstrate that his/her ability to
meet his/her academic responsibilities was seriously affected. An SMC can be
downloaded under the Medical Documentation heading of the following website:
https://studentservices.uwo.ca/secure/index.cfm.
ELECTRONIC AIDS:
The only electronic aids allowed during the exams are basic basic standalone calculators.
Computers, cell phones, PDAs, MP3 players or other digital devices are not allowed.
ONLINE INFORMATION:
The Office of the Registrar web site is: http://registrar.uwo.ca
The Student Development Services web site is: http://www.sdc.uwo.ca
The Psych Undergrad website: http://psychology.uwo.ca/undergradresponsibilities.htm
See the above websites for information on:
POLICY ON CHEATING AND ACADEMIC MISCONDUCT
PROCEDURES FOR APPEALING ACADEMIC EVALUATIONS
POLICY ON ATTENDANCE
POLICY ON MAKEUP EXAMS AND EXTENSIONS OF DEADLINES
POLICY FOR ASSIGNMENTS
SHORT ABSENCES
EXTENDED ABSENCES
DOCUMENTATION
ACADEMIC CONCERNS
CALENDAR REFERENCES
Psych/3840
LECTURE TOPICS AND ASSOCIATED READINGS
NOTE:
-Each reading’s online file name is underlined, but...
-Kline and Dick & Hagerty must now be purchased
from the bookstore due to copyright restrictions
-Note the skipped/skimmed pages where applicable
-The only formulas you need are on the Formula Sheet
1. INTRODUCTION
Kline, Chapter 1 - The characteristics of good tests in psychology. (skip pp. 17-23)
Chapter 2 - Making tests reliable I: Intelligence and ability. (skim)
Chapter 3 - Making tests reliable II: Personality inventories.
Chapter 4 - Making tests reliable III: Constructing other types of tests.
(skim; skip pp. 77-98)
Jackson, D. N. (1970). A sequential system for personality scale development.
In C. D. Spielberger (Ed.), Current topics in clinical and community psychology
(Vol. 2, pp. 61-96). New York: Academic Press. (pp. 72-96)
Paunonen, S. V., & Ashton, M. C. (2002). The nonverbal assessment of personality: The
NPQ and the FF-NPQ. In B. De Raad & M. Perugini (Eds.), Big Five assessment
(pp. 171-194). Göttingen, Germany: Hogrefe & Huber. (pp. 171-183)
2. CORRELATION, FACTOR ANALYSIS
Wiggins, J. S. (1973). Personality and prediction: Principles of personality assessment.
Reading, MA: Addison-Wesley. (pp. 6-30; skim formulas)
Edwards, A. L. (1970). The measurement of personality traits by scales and inventories.
New York: Holt, Rinehart, & Winston. (pp. 69-87; skim computations in
sections 6.7, 6.8, 6.10, 6.12, 6.14)
3. RELIABILITY
Kline, Chapter 1 - The characteristics of good tests in psychology. (skip pp. 17-23)
Chapter 5 - Computing test reliability. (skip pp. 126-128)
Allen, M. J., & Yen, W. M. (1979). Introduction to measurement theory. Monterey,
CA: Brooks/Cole. (pp. 56-70; Proofs are optional)
Dick, W., & Hagerty, W. (1971). Topics in measurement. New York: McGraw-Hill.
(pp. 1-56)
Paunonen, S. V. (1984). The reliability of aggregated measurements: Lessons
to be learned from psychometric theory. Journal of Research in Personality,
18, 383-385.
<<<<<<
MIDTERM EXAM
>>>>>>
Psych/3840
4. VALIDITY
Kline, Chapter 7 - Computing discriminatory power. (skip pp. 150-151)
Cronbach, L. J., & Meehl, P. E. (1955). Construct validity in psychological
tests. Psychological Bulletin, 52, 281-302. (skip pp. 290-299)
Campbell, D. T., & Fiske, D. W. (1959). Convergent and discriminant
validation by the multitrait-multimethod matrix. Psychological
Bulletin, 56, 81-105. (abridged)
5. ITEM ANALYSIS
Kline, Chapter 6 - Item trials.
Holden, R. R., & Jackson, D. N. (1979). Item subtlety and face validity in
personality assessment. Journal of Consulting and Clinical
Psychology, 47, 459-468.
Jackson, D. N. (1970). A sequential system for personality scale development.
In C. D. Spielberger (Ed.), Current topics in clinical and community psychology
(Vol. 2, pp. 61-96). New York: Academic Press. (pp. 72-96)
Paunonen, S. V., & Ashton, M. C. (2002). The nonverbal assessment of personality: The
NPQ and the FF-NPQ. In B. De Raad & M. Perugini (Eds.), Big Five assessment
(pp. 171-194). Göttingen, Germany: Hogrefe & Huber. (pp. 171-183)
6. STANDARDIZATION
Kline, Chapter 8 - Standardizing the test.
Seashore, H. G. (1955). Methods of expressing test scores. Test Service
Bulletin, No. 45. The Psychological Corporation. (pp. 7-10)
Ghiselli, E. E., Campbell, J. P., & Zedeck, S. (1981). Measurement theory
for the behavioral sciences. San Francisco: W. H. Freeman. (pp. 59-75)
7. COMPARISON OF METHODS
Kline, Chapter 9 - Other methods of test construction.
Nunnally, J. C. (1978). Psychometric theory. New York: McGraw-Hill.
(pp. 265-274)
Burisch, M. (1984). Approaches to personality inventory construction.
American Psychologist, 39, 214-227.
8. INTRODUCTION TO CAT AND IRT (optional)
MacDonald, P. (2002). Computer Adaptive Testing for measuring personality factors
using Item Response Theory. Ph.D dissertation, U.W.O. (pp. 1-34)