WESTERN UNIVERSITY LONDON CANADA Department of Psychology 2014-2015 PSYCHOLOGY 3840F - Section 001 RESEARCH IN TEST CONSTRUCTION 1.0 CALENDAR DESCRIPTION A review of methods for developing psychological tests, questionnaires, and surveys. Topics will include reliability, validity, norming, standardization, and strategies of test construction. Opportunities for developing measures of human abilities and personality will be provided. Strongly recommended for students planning graduate training in psychology and related disciplines. 3 lecture hours, 0.5 course. Antirequisites: Psychology 2080A/B. Antirequisites are courses that overlap sufficiently in content that only one can be taken for credit. So if you take a course that is an antirequisite to a course previously taken, you will lose credit for the earlier course, regardless of the grade achieved in the most recent course. Prerequisites: Minimum grade of 60% in both Psychology 2800E and 2810, or minimum grade of 70% in Psychology 2820E (or a minimum grade of 60% in both Psychology 2830A/B and 3830F/G at Huron). Unless you have either the requisites for this course or written special permission from your Dean to enroll in it, you may be removed from this course and it will be deleted from your record. This decision may not be appealed. You will receive no adjustment to your fees in the event that you are dropped from a course for failing to have the prerequisites. 2.0 COURSE INFORMATION Instructor: Lectures: Office Hours: Teaching Assistant: Office Hours: S. Paunonen (paunonen@uwo.ca; 519-661-2060) Thursdays, 2:30-5:30, Room 3120 SSC Mondays, 4:30-5:30, Room 6304 SSC Rachel Plouffe (rplouffe@uwo.ca) Wednesdays, 1:00-3:00, Room tba SSC If you or someone you know is experiencing distress, there are several resources here at Western to assist you. Please visit: http://www.uwo.ca/uwocom/mentalhealth/ for more information on these resources and on mental health. Please contact the course instructor if you require material in an alternate format or if you require any other arrangements to make this course more accessible to you. You may also wish to contact Services for Students with Disabilities (SSD) at 519-661-2111 (ext. 82147) for any specific questions regarding an accommodation. 3.0 TEXTBOOK Kline, P. (1986). A handbook of test construction. London: Methuen. Paunonen, S. V. (2012). Psychology 3840 lecture notes. UWO Bookstore. Psych/3840 4.0 COURSE OBJECTIVES The primary objective of this course is to introduce the student to the major methods of psychological test construction. Those methods will be examined in terms of their underlying psychometric theory related to reliability and validity estimation, standardization, and equating. Students will write and evaluate test items by constructing an in-class personality questionnaire. 5.0 EVALUATION Students will be required to write one 2-hour midterm exam and one 3-hour final exam. Exam items will be of the short answer, short essay, and problem-solving variety. The final exam will be cumulative in its content coverage. The graded midterm along with a complete exam scoring key can be examined with the TA. In addition to the two exams, a written project (2500 words) on item analysis will be assigned. Details about that project will be available on a separate handout. The Psychology Department does not require instructors to adjust their course grades to targets, but the expectation is that course marks will be distributed around the following averages: 70% 1000-level and 2000-level courses 72% 2100-level to 2990-level courses 75% 3000-level courses 80% 4000-level courses The Psychology Department follows these University of Western Ontario grading guidelines (see http://www.uwo.ca/univsec/pdf/academic_policies/general/grades_undergrad.pdf): A+ A B C D F 6.0 90-100 80-89 70-79 60-69 50-59 below 50 EXAM SCHEDULE Midterm Project Final 7.0 One could scarcely expect better from a student at this level Superior work that is clearly above average Good work, meeting all requirements, and eminently satisfactory Competent work, meeting requirements Fair work, minimally acceptable Fail DATE WEIGHT LOCATION Oct. 16 Nov. 27 Dec. 6-17 .30 .30 (-5% p.d. late) .40 EC 2155 6304 SSC tba LECTURE TOPICS AND ASSOCIATED READINGS The lecture topics and the assigned readings are listed on a separate page. The textbook and all readings can be found online. The Lecture Notes are sold through the Bookstore. 8.0 STATEMENT ON ACADEMIC OFFENCES Students are responsible for understanding the nature and avoiding the occurrence of plagiarism and other scholastic offenses. Plagiarism and cheating are considered very serious offenses because they undermine the integrity of research and education. Actions constituting a scholastic offense are described at the following link: http://www.uwo.ca/univsec/pdf/academic_policies/appeals/appealsundergrad.pdf Psych/3840 8.0 STATEMENT ON ACADEMIC OFFENCES (continued) As of Sept. 1, 2009, the Department of Psychology will take the following steps to detect scholastic offenses. (a) All multiple-choice tests and exams will be checked for similarities in the pattern of responses using reliable software, and records will be made of student seating locations in all tests and exams. (b) All written assignments will be submitted to TurnItIn, a service designed to detect and deter plagiarism by comparing written material to over 5 billion pages of content located on the Internet or in TurnItIn’s databases. (c) All papers submitted for such checking will be included as source documents in the reference database for the purpose of detecting plagiarism of papers subsequently submitted to the system. Use of the service is subject to the licensing agreement, currently between The University of Western Ontario and Turnitin.com (http://www.turnitin.com). Possible penalties for a scholastic offense include failure of the assignment, failure of the course, suspension from the University, and expulsion from the University. 9.0 OTHER INFORMATION The information below pertains to make-up exams, electronic aids, online resources, etc. MAKE-UP EXAMS: 1. Only under special circumstances will a student be allowed to write a make-up test or exam or be given an extension on an assignment. These include medical or compassionate reasons substantiated by the proper documentation (see below) approved by your faculty’s Academic Counselling Office. 2. If the missed exam is the final examination (set by the Registrar’s Office) the student must contact the Academic Counselling Office to request a permission form for writing a Special Final Examination. That form must be signed by the instructor, the department chairperson, and the Dean. 3. A student missing an exam or assignment must notify the instructor on the day of the exam or deadline, or on the following day at the latest. 4. The student should try to notify the instructor in person or by telephone, rather than by email. If leaving a phone mail message, be sure to leave your telephone number. 5. A student who misses an exam or deadline for other than medical or compassion reasons, or who is unable to substantiate a claim, or who fails to notify the instructor by the day following the exam or deadline will be assigned a grade of zero for that evaluation. Late written assignments will be assessed a penalty of 5% per day. 6. Make-up exams or extensions to deadlines will not be scheduled to accommodate students’ travel or work plans. MEDICAL NOTES: In May, 2008, The University of Western Ontario’s Senate approved a new medical note policy, which affects all students. The following is an outline of that policy. For more detailed information and forms, please visit: https://studentservices.uwo.ca/secure/index.cfm For further policy information please visit: http://www.uwo.ca/univsec/handbook/appeals/accommodation_medical.pdf Psych/3840 9.0 OTHER INFORMATION (continued) MEDICAL NOTES: (cont.) Notes from Family Physicians and Walk-In Clinics A Western Student Medical Certificate (SMC) is required where a student is seeking academic accommodation. This documentation should be obtained at the time of the initial consultation with the physician or walk-in clinic. An SMC can be downloaded under the Medical Documentation heading of the following website: https://studentservices.uwo.ca/secure/index.cfm. The completed SMC should be returned to the student’s home Faculty Academic Counselling Service. Notes from Student Health Services Students must request an SMC be filled out by the attending physician. There is a charge for this service. The completed SMC should be returned to the student’s home Faculty Academic Counselling Service. Notes from Hospital Urgent Care Centres or Emergency Departments Students should request that an SMC be filled out. Students may bring this form with them, or request alternative Emergency Department documentation. Documentation should be secured at the time of the initial visit to the Emergency Department. Where it is not possible for a student to have an SMC completed by the attending physician, the student must request documentation sufficient to demonstrate that his/her ability to meet his/her academic responsibilities was seriously affected. An SMC can be downloaded under the Medical Documentation heading of the following website: https://studentservices.uwo.ca/secure/index.cfm. ELECTRONIC AIDS: The only electronic aids allowed during the exams are basic basic standalone calculators. Computers, cell phones, PDAs, MP3 players or other digital devices are not allowed. ONLINE INFORMATION: The Office of the Registrar web site is: http://registrar.uwo.ca The Student Development Services web site is: http://www.sdc.uwo.ca The Psych Undergrad website: http://psychology.uwo.ca/undergradresponsibilities.htm See the above websites for information on: POLICY ON CHEATING AND ACADEMIC MISCONDUCT PROCEDURES FOR APPEALING ACADEMIC EVALUATIONS POLICY ON ATTENDANCE POLICY ON MAKEUP EXAMS AND EXTENSIONS OF DEADLINES POLICY FOR ASSIGNMENTS SHORT ABSENCES EXTENDED ABSENCES DOCUMENTATION ACADEMIC CONCERNS CALENDAR REFERENCES Psych/3840 LECTURE TOPICS AND ASSOCIATED READINGS NOTE: -Each reading’s online file name is underlined, but... -Kline and Dick & Hagerty must now be purchased from the bookstore due to copyright restrictions -Note the skipped/skimmed pages where applicable -The only formulas you need are on the Formula Sheet 1. INTRODUCTION Kline, Chapter 1 - The characteristics of good tests in psychology. (skip pp. 17-23) Chapter 2 - Making tests reliable I: Intelligence and ability. (skim) Chapter 3 - Making tests reliable II: Personality inventories. Chapter 4 - Making tests reliable III: Constructing other types of tests. (skim; skip pp. 77-98) Jackson, D. N. (1970). A sequential system for personality scale development. In C. D. Spielberger (Ed.), Current topics in clinical and community psychology (Vol. 2, pp. 61-96). New York: Academic Press. (pp. 72-96) Paunonen, S. V., & Ashton, M. C. (2002). The nonverbal assessment of personality: The NPQ and the FF-NPQ. In B. De Raad & M. Perugini (Eds.), Big Five assessment (pp. 171-194). Göttingen, Germany: Hogrefe & Huber. (pp. 171-183) 2. CORRELATION, FACTOR ANALYSIS Wiggins, J. S. (1973). Personality and prediction: Principles of personality assessment. Reading, MA: Addison-Wesley. (pp. 6-30; skim formulas) Edwards, A. L. (1970). The measurement of personality traits by scales and inventories. New York: Holt, Rinehart, & Winston. (pp. 69-87; skim computations in sections 6.7, 6.8, 6.10, 6.12, 6.14) 3. RELIABILITY Kline, Chapter 1 - The characteristics of good tests in psychology. (skip pp. 17-23) Chapter 5 - Computing test reliability. (skip pp. 126-128) Allen, M. J., & Yen, W. M. (1979). Introduction to measurement theory. Monterey, CA: Brooks/Cole. (pp. 56-70; Proofs are optional) Dick, W., & Hagerty, W. (1971). Topics in measurement. New York: McGraw-Hill. (pp. 1-56) Paunonen, S. V. (1984). The reliability of aggregated measurements: Lessons to be learned from psychometric theory. Journal of Research in Personality, 18, 383-385. <<<<<< MIDTERM EXAM >>>>>> Psych/3840 4. VALIDITY Kline, Chapter 7 - Computing discriminatory power. (skip pp. 150-151) Cronbach, L. J., & Meehl, P. E. (1955). Construct validity in psychological tests. Psychological Bulletin, 52, 281-302. (skip pp. 290-299) Campbell, D. T., & Fiske, D. W. (1959). Convergent and discriminant validation by the multitrait-multimethod matrix. Psychological Bulletin, 56, 81-105. (abridged) 5. ITEM ANALYSIS Kline, Chapter 6 - Item trials. Holden, R. R., & Jackson, D. N. (1979). Item subtlety and face validity in personality assessment. Journal of Consulting and Clinical Psychology, 47, 459-468. Jackson, D. N. (1970). A sequential system for personality scale development. In C. D. Spielberger (Ed.), Current topics in clinical and community psychology (Vol. 2, pp. 61-96). New York: Academic Press. (pp. 72-96) Paunonen, S. V., & Ashton, M. C. (2002). The nonverbal assessment of personality: The NPQ and the FF-NPQ. In B. De Raad & M. Perugini (Eds.), Big Five assessment (pp. 171-194). Göttingen, Germany: Hogrefe & Huber. (pp. 171-183) 6. STANDARDIZATION Kline, Chapter 8 - Standardizing the test. Seashore, H. G. (1955). Methods of expressing test scores. Test Service Bulletin, No. 45. The Psychological Corporation. (pp. 7-10) Ghiselli, E. E., Campbell, J. P., & Zedeck, S. (1981). Measurement theory for the behavioral sciences. San Francisco: W. H. Freeman. (pp. 59-75) 7. COMPARISON OF METHODS Kline, Chapter 9 - Other methods of test construction. Nunnally, J. C. (1978). Psychometric theory. New York: McGraw-Hill. (pp. 265-274) Burisch, M. (1984). Approaches to personality inventory construction. American Psychologist, 39, 214-227. 8. INTRODUCTION TO CAT AND IRT (optional) MacDonald, P. (2002). Computer Adaptive Testing for measuring personality factors using Item Response Theory. Ph.D dissertation, U.W.O. (pp. 1-34)