Mastering Modern Psychological Testing: Theory & Methods

Mastering Modern Psychological Testing Cecil R. Reynolds • Robert A. Altmann Daniel N. Allen Mastering Modern Psychological Testing Theory and Methods Second Edition Cecil R. Reynolds Austin, TX, USA Robert A. Altmann Minneapolis, MN, USA Daniel N. Allen Department of Psychology University of Nevada, Las Vegas Las Vegas, NV, USA First edition published by Pearson Education. Authors have written release from Pearson to publish a new edition elsewhere. ISBN 978-3-030-59454-1 ISBN 978-3-030-59455-8 https://doi.org/10.1007/978-3-030-59455-8 (eBook) © Springer Nature Switzerland AG 2012, 2021 This work is subject to copyright. All rights are reserved by the Publisher, whether the whole or part of the material is concerned, specifically the rights of translation, reprinting, reuse of illustrations, recitation, broadcasting, reproduction on microfilms or in any other physical way, and transmission or information storage and retrieval, electronic adaptation, computer software, or by similar or dissimilar methodology now known or hereafter developed. The use of general descriptive names, registered names, trademarks, service marks, etc. in this publication does not imply, even in the absence of a specific statement, that such names are exempt from the relevant protective laws and regulations and therefore free for general use. The publisher, the authors, and the editors are safe to assume that the advice and information in this book are believed to be true and accurate at the date of publication. Neither the publisher nor the authors or the editors give a warranty, expressed or implied, with respect to the material contained herein or for any errors or omissions that may have been made. The publisher remains neutral with regard to jurisdictional claims in published maps and institutional affiliations. This Springer imprint is published by the registered company Springer Nature Switzerland AG The registered company address is: Gewerbestrasse 11, 6330 Cham, Switzerland Contents 1Introduction to Psychological Assessment �� 1 1.1Brief History of Testing�� 6 1.1.1Earliest Testing: Circa 2200 BC�� 6 1.1.2Eighteenth- and Nineteenth-Century Testing�� 6 1.1.3Brass Instruments Era �� 7 1.1.4Twentieth-Century Testing�� 8 1.1.5Twenty-First-Century Testing �� 10 1.2The Language of Assessment�� 11 1.2.1Tests�� 11 1.2.2Standardized Tests�� 11 1.2.3Measurement�� 11 1.2.4Assessment�� 12 1.2.5Are Tests, Measurement, and Assessment Interchangeable Terms?�� 13 1.2.6Other Important Terms�� 13 1.3Types of Tests�� 14 1.3.1Maximum Performance Tests �� 15 1.3.2Typical Response Tests �� 17 1.4Types of Scores �� 19 1.5Assumptions of Psychological Assessment�� 21 1.5.1Assumption #1: Psychological Constructs Exist�� 21 1.5.2Assumption #2: Psychological Constructs Can Be Measured�� 21 1.5.3Assumption #3: Although We Can Measure Constructs, Our Measurement Is Not Perfect �� 21 1.5.4Assumption #4: There Are Different Ways to Measure Any Given Construct�� 22 1.5.5Assumption #5: All Assessment Procedures Have Strengths and Limitations�� 22 1.5.6Assumption #6: Multiple Sources of Information Should Be Part of the Assessment Process �� 22 1.5.7Assumption #7: Performance on Tests Can Be Generalized to Non-Test Behaviors�� 23 v vi Contents 1.5.8Assumption #8: Assessment Can Provide Information that Helps Psychologists Make Better Professional Decisions�� 23 1.5.9Assumption #9: Assessments Can Be Conducted in a Fair Manner �� 23 1.5.10Assumption #10: Testing and Assessment Can Benefit Individuals and Society as a Whole�� 24 1.6Why Use Tests?�� 26 1.7Common Applications of Psychological Assessments �� 28 1.7.1Diagnosis�� 28 1.7.2Treatment Planning and Treatment Effectiveness�� 30 1.7.3Selection, Placement, and Classification�� 30 1.7.4Self-Understanding �� 31 1.7.5Evaluation �� 31 1.7.6Licensing�� 31 1.7.7Program Evaluation�� 31 1.7.8Scientific Method�� 32 1.8Common Criticisms of Testing and Assessment�� 32 1.9Participants in the Assessment Process�� 34 1.9.1People Who Develop Tests �� 34 1.9.2People Who Use Tests�� 37 1.9.3People Who Take Tests �� 38 1.9.4Other People Involved in Assessment Process�� 39 1.10Psychological Assessment in the Twenty-First Century �� 39 1.10.1Computerized Adaptive Testing (CAT)�� 39 1.10.2Other Technological Applications Used in Assessment�� 39 1.10.3“Authentic” Assessments�� 40 1.10.4Health-Care Delivery Systems�� 41 1.10.5High-Stakes Assessment �� 41 1.11Summary �� 43 References�� 45 2The Basic Statistics of Measurement�� 49 2.1Scales of Measurement �� 51 2.1.1What Is Measurement? �� 51 2.1.2Nominal Scales �� 51 2.1.3Ordinal Scales �� 52 2.1.4Interval Scales�� 52 2.1.5Ratio Scales�� 54 2.2The Description of Test Scores �� 56 2.2.1Distributions�� 56 2.2.2Measures of Central Tendency�� 60 2.2.3Measures of Variability �� 66 2.2.4The Normal Distribution�� 70 2.3Correlation Coefficients�� 73 Contents vii 2.3.1Scatterplots�� 76 2.3.2Types of Correlation Coefficients �� 78 2.3.3Factors that Affect Correlation Coefficients �� 80 2.3.4Correlation Versus Causation�� 81 2.4Linear Regression �� 83 2.4.1Standard Error of Estimate�� 84 2.5Summary �� 86 Practice Items�� 88 References�� 89 3The Meaning of Test Scores�� 91 3.1Norm-Referenced and Criterion-Referenced Score Interpretations �� 93 3.1.1Norm-Referenced Interpretations �� 94 3.1.2Criterion-Referenced Interpretations�� 111 3.1.3Norm-Referenced, Criterion-Referenced, or Both?�� 114 3.2Scores Based on Item Response Theory �� 115 3.3So What Scores Should We Use: Norm-Referenced, Criterion-Referenced, or Rasch-Based Scores?�� 117 3.4Qualitative Description of Test Scores�� 119 3.5Reporting Information on Normative Samples and Test Scores �� 120 3.6Summary �� 125 Practice Items�� 130 References�� 130 4Reliability�� 133 4.1Classical Test Theory and Measurement Error �� 135 4.2Sources of Measurement Error �� 138 4.2.1Content Sampling Error�� 138 4.2.2Time Sampling Error�� 139 4.2.3Other Sources of Error�� 140 4.3Reliability Coefficients �� 140 4.3.1Test-Retest Reliability�� 142 4.3.2Alternate-Form Reliability�� 144 4.3.3Internal-Consistency Reliability �� 145 4.3.4Inter-Rater Reliability �� 148 4.3.5Reliability Estimates Are Not Independent�� 151 4.3.6Reliability of Composite Scores �� 152 4.3.7Reliability of Difference Scores�� 152 4.3.8Selecting a Reliability Coefficient�� 155 4.3.9Evaluating Reliability Coefficients �� 156 4.3.10How to Improve Reliability�� 158 4.3.11Special Problems in Estimating Reliability�� 160 4.4The Standard Error of Measurement�� 164 4.4.1Evaluating the Standard Error of Measurement�� 165 4.4.2Calculating Confidence Intervals�� 167 viii Contents 4.5Modern Test Theories �� 169 4.5.1Generalizability Theory�� 169 4.5.2Item Response Theory�� 171 4.6Reporting Reliability Information�� 172 4.6.1How Test Manuals Report Reliability Information: The Reynolds Intellectual Assessment Scales, Second Edition (RIAS-2)�� 173 4.7Reliability: Practical Strategies for Educators�� 177 4.8Summary �� 179 Practice Items�� 181 References�� 182 5Validity�� 185 5.1Threats to Validity �� 188 5.1.1Examinee Characteristics�� 189 5.1.2Test Administration and Scoring Procedures�� 189 5.1.3Instruction and Coaching�� 189 5.2Reliability and Validity �� 190 5.3“Types of Validity” Versus “Types of Validity Evidence”�� 191 5.4Sources of Validity Evidence�� 193 5.4.1Evidence Based on Test Content�� 193 5.4.2Evidence Based on Response Processes �� 197 5.4.3Evidence Based on Internal Structure�� 198 5.4.4Evidence Based on Relations to Other Variables�� 203 5.4.5Evidence Based on Consequences of Testing �� 213 5.5Integrating Evidence of Validity �� 214 5.6How Test Manuals Report Validity Evidence: The Reynolds Intellectual Assessment Scales, Second Edition (RIAS–2)�� 216 5.7Summary �� 218 References�� 220 6Item Development�� 223 6.1Item Formats �� 225 6.2General Item Writing Guidelines�� 230 6.3Maximum-Performance Tests �� 235 6.3.1Multiple-Choice Items�� 236 6.3.2True–False Items�� 241 6.3.3Matching Items �� 244 6.3.4Essay Items �� 246 6.3.5Short-Answer Items�� 248 6.4Typical-Response Tests�� 250 6.4.1Typical-Response Item Formats�� 250 6.4.2Typical-Response Item Guidelines �� 258 6.5Summary �� 259 References�� 261 Contents ix 7Item Analysis: Methods for Fitting the Right Items to the Right Test �� 263 7.1Item Difficulty Index (or Item Difficulty Level)�� 265 7.1.1Special Assessment Situations and Item Difficulty�� 267 7.2Item Discrimination�� 269 7.2.1Discrimination Index�� 269 7.2.2Item Discrimination on Mastery Tests�� 272 7.2.3Item Discrimination on Typical-Response Tests�� 274 7.2.4Difficulty and Discrimination on Speed Tests�� 275 7.2.5Examples of Item Difficulty and Discrimination Indices �� 276 7.3Distracter Analysis�� 277 7.3.1How Distracters Influence Item Difficulty and Discrimination�� 278 7.4Qualitative Item Analysis�� 279 7.5Item Characteristic Curves and Item Response Theory�� 280 7.5.1Item Characteristic Curves�� 281 7.5.2IRT Models �� 284 7.5.3Invariance of Item Parameters�� 284 7.5.4Special Applications of IRT�� 286 7.6Summary �� 287 References�� 289 8Achievement Tests in the Era of High-Stakes Assessment�� 291 8.1The Impetus for Achievement Tests�� 294 8.2Group-Administered Achievement Tests�� 299 8.2.1Commercial Standardized Achievement Test �� 301 8.2.2State-Developed Achievement Tests �� 305 8.2.3Best Practices in Preparing Students for Standardized Assessment�� 312 8.3Individual Achievement Tests �� 317 8.3.1Wechsler Individual Achievement Test, Third Edition (WIAT-III)�� 317 8.3.2Woodcock-Johnson IV Tests of Achievement (WJ-IV ACH)�� 318 8.3.3Wide Range Achievement Test Fifth Edition (WRAT5)�� 319 8.3.4Individual Achievement Tests That Focus on Specific Skills�� 319 8.4Selecting an Achievement Battery�� 320 8.5Teacher-Constructed Achievement Tests and Student Evaluation�� 320 8.6Achievement Tests: Not Only in the Public Schools! �� 325 8.6.1Examination for Professional Practice in Psychology (EPPP)�� 325 8.6.2United States Medical Licensing Examination (USMLE)�� 326 8.7Summary �� 327 References�� 329 x Contents 9Assessment of Intelligence�� 331 9.1A Brief History of Intelligence Tests�� 336 9.2The Use of Aptitude and Intelligence Tests in School Settings�� 342 9.2.1Aptitude-Achievement Discrepancies�� 343 9.2.2A New Assessment Strategy for Specific Learning Disabilities: Response to Intervention (RTI)�� 345 9.2.3Diagnosing Intellectual Disability�� 347 9.3The Use of Aptitude and Intelligence Tests in Clinical Settings�� 347 9.4Major Aptitude/Intelligence Tests�� 348 9.4.1Group Aptitude/Intelligence Tests�� 348 9.4.2Individual Aptitude/Intelligence Tests�� 353 9.5Selecting Aptitude/Intelligence Tests�� 364 9.6Understanding the Report of an Intellectual Assessment �� 365 9.7Summary �� 378 References�� 380 10Assessment of Personality �� 383 10.1Assessing Personality�� 385 10.1.1Response Sets and Dissimulation �� 391 10.1.2Factors Affecting Reliability and Validity�� 394 10.2Objective Personality Tests: An Overview�� 396 10.2.1Content/Rational Approach�� 397 10.2.2Empirical Criterion Keying�� 398 10.2.3Factor Analysis�� 408 10.2.4Theoretical Approach�� 411 10.3Assessment of Personality in Children and Adolescents�� 414 10.3.1Behavior Assessment System for Children, Third Edition: Self-Report of Personality (SRP)�� 414 10.3.2Single-Domain Self-Report Measures�� 417 10.4Projective Personality Tests: An Overview �� 418 10.4.1Projective Drawings�� 419 10.4.2Sentence Completion Tests �� 420 10.4.3Apperception Tests�� 420 10.4.4Inkblot Techniques�� 421 10.5Summary �� 422 References�� 425 11Behavioral Assessment�� 427 11.1Assessing Behavior �� 430 11.2Response Sets�� 431 11.3Assessment of Behavior in the Schools�� 433 11.4Behavioral Interviewing�� 435 11.5Behavior Rating Scales �� 436 11.5.1Behavior Assessment System for Children, Third Edition: Teacher Rating Scales and Parent Rating Scales (TRSs and PRSs)�� 438 Contents xi 11.5.2Achenbach System of Empirically Based Assessment: Child Behavior Checklist and Teacher Report Form (CBCL and TRF)�� 444 11.5.3Single-Domain or Syndrome-Specific Rating Scales �� 446 11.5.4Adaptive Behavior Scales �� 447 11.5.5Adult Behavior Rating Scales�� 448 11.6Direct Observation�� 450 11.7Continuous Performance Tests�� 454 11.8Psychophysiological Assessments�� 455 11.9Summary �� 456 References�� 458 12Employment and Vocational Testing�� 461 12.1Historical View of I/O Psychology �� 463 12.2Personnel Selection Approaches �� 466 12.2.1Cognitive Ability�� 466 12.2.2Interviews�� 469 12.2.3Integrity�� 474 12.2.4Assessment Centers�� 476 12.2.5Work Sample Tests�� 477 12.2.6Biodata�� 478 12.3Choosing a Personnel Selection Approach �� 479 12.3.1Advantages and Disadvantages of Different Approaches �� 479 12.3.2Applicant Reactions�� 481 12.3.3Job Analysis�� 482 12.4Evaluating Job Performance �� 483 12.4.1Approaches to Performance Ratings�� 484 12.4.2Comparison of Rating Approaches �� 485 12.4.3Types of Rating Methods�� 486 12.4.4Sources of Error�� 487 12.5Legal Issues�� 488 12.5.1The Uniform Guidelines on Employee Selection Procedures (1978) �� 489 12.5.2Principles for the Validation and Use of Personnel Selection Procedures, Fifth Edition (2018)�� 490 12.6Career Assessment�� 491 12.6.1Strong Interest Inventory, Revised Edition �� 491 12.6.2Career Decision-Making System, Revised �� 492 12.6.3Self-Directed Search �� 493 12.7Summary �� 493 References�� 494 13Neuropsychological Testing�� 499 13.1Components of a Neuropsychological Evaluation�� 502 13.2Neuropsychological Assessment Approaches and Instruments�� 506 xii Contents 13.2.1The Halstead-Reitan Neuropsychological Test Battery (HRNB)�� 507 13.2.2The Luria-Nebraska Neuropsychological Battery (LNNB) for Adults �� 509 13.2.3The Boston Process Approach�� 510 13.3Assessment of Memory Functions�� 512 13.3.1TOMAL-2: An Example of a Contemporary Comprehensive Memory Assessment �� 515 13.4The Process of Neuropsychological Assessment�� 518 13.4.1Referral �� 519 13.4.2Review of Records�� 522 13.4.3Clinical Interview�� 522 13.4.4Test Selection�� 524 13.4.5Test Conditions �� 528 13.5Measurement of Deficits and Strengths�� 531 13.5.1Normative Approach �� 531 13.5.2Deficit Measurement Approach�� 532 13.5.3Premorbid Ability �� 534 13.5.4Pattern Analysis�� 536 13.5.5Pathognomonic Signs�� 536 13.6Summary �� 537 References�� 538 14Forensic Applications of Psychological Assessment�� 543 14.1What Is Forensic Psychology?�� 545 14.2Expert Witnesses and Expert Testimony�� 547 14.3Clinical Assessment Versus Forensic Assessment�� 548 14.4Applications in Criminal Proceedings�� 550 14.4.1Not Guilty by Reason of Insanity: The NGRI Defense�� 551 14.4.2Competency to Stand Trial�� 552 14.4.3Transfer of a Juvenile to Adult Criminal Court�� 554 14.4.4Mitigation in Sentencing�� 554 14.4.5The Special Case of Intellectual Disability in Capital Sentencing�� 556 14.4.6Competency to Be Executed �� 558 14.5Applications in Civil Proceedings�� 559 14.5.1Personal Injury Litigation �� 559 14.5.2Divorce and Child Custody�� 560 14.5.3Determining Common Civil Competencies�� 563 14.5.4Other Civil Matters �� 564 14.6Third Party Observers in Forensic Psychological Testing�� 564 14.7Detection of Malingering and Other Forms of Dissimulation�� 567 14.8The Admissibility of Testimony Based on Psychological Testing Results�� 568 14.9Summary �� 570 References�� 571 Contents xiii 15The Problem of Bias in Psychological Assessment�� 573 15.1What Do We Mean by Bias?�� 577 15.2Past and Present Concerns: A Brief Look�� 578 15.3The Controversy Over Bias in Testing: Its Origin, What It Is, and What It Is Not�� 579 15.3.1Explaining Mean Group Differences�� 582 15.3.2Test Bias and Etiology�� 583 15.3.3Test Bias and Fairness�� 583 15.3.4Test Bias and Offensiveness�� 585 15.3.5Test Bias and Inappropriate Test Administration and Use�� 585 15.3.6Bias and Extraneous Factors�� 586 15.4Cultural Bias and the Nature of Psychological Testing�� 586 15.5Objections to the Use of Educational and Psychological Tests with Minority Students�� 587 15.5.1Inappropriate Content �� 588 15.5.2Inappropriate Standardization Samples�� 588 15.5.3Examiner and Language Bias �� 588 15.5.4Inequitable Social Consequences�� 589 15.5.5Measurement of Different Constructs�� 589 15.5.6Differential Predictive Validity �� 589 15.5.7Qualitatively Distinct Aptitude and Personality�� 589 15.6The Problem of Definition in Test Bias Research: Differential Validity�� 592 15.7Cultural Loading, Cultural Bias, and Culture-Free Tests�� 594 15.8Inappropriate Indicators of Bias: Mean Differences and Equivalent Distributions�� 595 15.9Bias in Test Content�� 596 15.9.1How Test Publishers Commonly Identify Biased Items �� 598 15.10Bias in Other Internal Features of Tests�� 601 15.10.1How Test Publishers Commonly Identify Bias in Construct Measurement�� 603 15.11Bias in Prediction and in Relation to Variables External to the Test �� 604 15.11.1How Test Publishers Commonly Identify Bias in Prediction�� 609 15.12Summary �� 609 References�� 610 16Assessment Accommodations�� 615 16.1Accommodations Versus Modifications�� 619 16.2The Rationale for Assessment Accommodations�� 619 16.3When Are Accommodations Not Appropriate or Necessary?�� 620 16.4Strategies for Accommodations�� 621 16.4.1Modifications of Presentation Format�� 622 16.4.2Modifications of Response Format �� 623 xiv Contents 16.4.3Modifications of Timing �� 623 16.4.4Modifications of Setting�� 624 16.4.5Adaptive Devices and Supports�� 625 16.4.6Using Only Portions of a Test �� 625 16.4.7Using Alternate Assessments�� 626 16.5Determining What Accommodations to Provide�� 627 16.6Assessment of English Language Learners (ELLs)�� 632 16.7Reporting Results of Modified Assessments�� 634 16.8Summary �� 637 References�� 639 17Best Practices: Legal and Ethical Issues�� 641 17.1Guidelines for Developing Assessments�� 643 17.2Guidelines for Selecting Published Assessments�� 645 17.3Guidelines for Administering Assessments�� 650 17.4Guidelines for Scoring Assessments�� 652 17.5Guidelines for Interpreting Assessment Results, Making Clinical Decisions, and Reporting Results�� 654 17.6Responsibilities of Test Takers�� 658 17.7Summary and Top 10 Assessment-Related Behaviors to Avoid �� 661 References�� 661 18How to Develop a Psychological Test: A Practical Approach�� 663 18.1Phase I: Test Conceptualization�� 665 18.1.1Conduct a Literature Review and Develop a Statement of Need for the Test�� 665 18.1.2Describe the Proposed Uses and Interpretations of Results From the Test �� 667 18.1.3Determine Who Will Use the Test and Why �� 668 18.1.4Develop Conceptual and Operational Definitions of Constructs You Intend to Measure�� 669 18.1.5Determine Whether Measures of Dissimulation Are Needed and If So, What Kind�� 670 18.2Phase II: Specification of Test Structure and Format�� 675 18.2.1Designate the Age Range Appropriate for the Measure�� 675 18.2.2Determine and Describe the Testing Format�� 676 18.2.3Describe the Structure of the Test �� 676 18.2.4Develop a Table of Specifications (TOS)�� 676 18.2.5Determine and Describe the Item Formats and Write Instructions for Administration and Scoring�� 680 18.2.6Develop an Explanation of Methods for Item Development, Tryout, and Final Item Selection �� 682 18.3Phase III: Planning Standardization and Psychometric Studies �� 684 18.3.1Specify a Sampling Plan for Standardization �� 684 18.3.2Determine Your Choice of Scaling Methods and Rationale�� 685 Contents xv 18.3.3Briefly Outline the Reliability Studies to Be Performed and Their Rationale �� 688 18.3.4Briefly Outline the Validity Studies to Be Performed and Their Rationale�� 688 18.3.5Determine If There Are Any Special Studies That May Be Needed for Development of This Test or to Support Proposed Interpretations of Performance�� 689 18.3.6List the Components of the Test �� 690 18.4Phase 4: Plan Implementation�� 690 18.4.1Reevaluate the Test Content and Structure �� 690 18.4.2Prepare the Test Manual�� 690 18.4.3Submit a Test Proposal�� 691 18.5Summary �� 691 References�� 692 Appendix: Calculation (Table A.1) �� 693 Index�� 701 List of Figures Fig. 2.1 Fig. 2.2 Fig. 2.3 Fig. 2.4 Fig. 2.5 Fig. 2.6 Histogram graph of the homework scores�� 58 Hypothetical distribution of large standardization sample�� 59 Negatively skewed distribution�� 59 Positively skewed distribution�� 60 Bimodal distribution�� 63 Relationship between mean, median, and mode in normal and skewed distributions�� 65 Fig. 2.7 Three distributions with the same mean but different degrees of variability�� 67 Fig. 2.8 Normal distribution with mean, standard deviation, and percentages�� 71 Fig. 2.9 Scatterplot of different correlation coefficients�� 77 Fig. 2.10 Curve to fit a scatterplot of curvilinear data�� 80 Fig. 2.11 Example of a graph of the equation of a straight line, also known as a regression line or prediction line. Note: Y = a + bX when a = 10 and b = 0.5. For example, if X is 30, then Y = 10 + (0.5) 30 = 25�� 84 Fig. 3.1 Fig. 3.2 Normal distribution illustrating the relationship among standard scores�� 109 Example of RIAS-2 record form. (Source. Reynolds, C. R., & Kamphaus, R. W., Reynolds Intellectual Assessment Scales (2nd ed.), RIAS–2 Score Report p. 2–3, p. 18, © Psychological Assessment Resources. Reproduced with permission)�� 126 Fig. 4.1 Fig. 4.2 Partitioning the variance to reflect sources of variance�� 171 Test information function (TIF) and standard errors (Osterlind, Figure 10.13/p. 293)�� 172 Fig. 5.1 Fig. 5.2 Fig. 5.3 Fig. 5.4 Illustration of item relevance�� 195 Illustration of content coverage�� 196 Scree plot�� 201 Illustration of predictive and concurrent studies�� 204 xvii xviii List of Figures Fig. 7.1 Fig. 7.2 Fig. 7.3 Fig. 7.4 Fig. 7.5 Typical ICC�� 281 ICC illustrating difficulty and discrimination�� 282 Two ICCs with the same discrimination but different difficulties�� 282 Two ICCs with the same difficulty but different discrimination�� 283 ICCs for a biased item�� 285 Fig. 9.1 A continuum of general abilities. Note. Modeled after Anastasi and Urbina (1997), Cronbach (1990), and others�� 333 Cattell-Horn-Carroll (CHC) hierarchical model of intelligence�� 360 RIAS-2 subtest scores, index scores, and age-adjusted T scores�� 370 RIAS-2 discrepancy score summary table�� 371 RIAS-2 profiles�� 372 RIAS-2 total battery profiles�� 373 RIAS-2 extended score summary table�� 376 Fig. 9.2 Fig. 9.3 Fig. 9.4 Fig. 9.5 Fig. 9.6 Fig. 9.7 Fig. 10.1 MMPI-2 profile with a 4–9 code type. MMPI-2 profile: excerpted from the MMPI-2 (Minnesota Multiphasic Personality Inventory-2) Manual for Administration, Scoring, and Interpretation, Revised Edition. Copyright © 2001 by the Regents of the University of Minnesota. (Used with permission of the University of Minnesota Press. All rights reserved. “MMPI®” and “Minnesota Multiphasic Personality Inventory®” are trademarks owned by the Regents of the University of Minnesota)�� 400 Fig. 10.2 BASC-3 Self-Report of Personality Profile. Behavior Assessment System for Children, Third Edition Copyright ©2015 NCS Pearson, Inc. (Reproduced with permission. All rights reserved)�� 416 Fig. 10.3 Inkblots similar to those used in the Rorschach�� 422 Fig. 11.1 An example of a completed TRS profile�� 442 Fig. 15.1 An example of an item characteristic curve or ICC�� 599 Fig. 15.2 A visual representation of DIF is the region between group A’s and group B’s item characteristic curves (ICCs)�� 600 Fig. 15.3 Equal slopes and intercepts. Note: Equal slopes and intercepts result in homogeneity of regression where the regression lines for different groups are the same�� 605 Fig. 15.4 Equal slopes with differing intercepts. Note: Equal slopes with differing intercepts result in parallel regression lines that produce a constant bias in prediction�� 606 Fig. 15.5 Equal intercepts and differing slopes. Note: Equal intercepts and differing slopes result in nonparallel regression lines, with the degree of bias depending on the distance of the individual’s score from the origin�� 607 Fig. 15.6 Differing Slopes and Intercepts. Note: Differing slopes and intercepts result in a complex situation where the amount and the direction of the bias are a function of the distance of an individual’s score from the origin�� 608 List of Tables Table 1.1 Table 1.2 Table 1.3 Table 1.4 Table 1.5 Milestones in testing history�� 10 Major categories of tests�� 18 Norm-and criterion-referenced scores�� 20 Assumptions of psychological assessment�� 24 Major applications of psychological assessments�� 29 Table 2.1 Table 2.2 Table 2.3 Table 2.4 Table 2.5 Table 2.6 Table 2.7 Table 2.8 Common nominal, ordinal, interval, and ratio scales�� 55 Distribution of scores for 20 students�� 57 Ungrouped frequency distribution�� 57 Grouped frequency distribution�� 58 Calculating the standard deviation and variance�� 69 Calculating a Pearson correlation coefficient�� 79 Calculating linear regression�� 85 Calculating the standard error of the estimate�� 86 Table 3.1 Table 3.2 Table 3.3 Table 3.4 Transforming raw scores to standard scores�� 101 Relationship of different standard score formats�� 104 Converting standard scores from one format to another�� 105 Characteristics of norm-referenced and criterion-referenced scores�� 115 Comparison of U.S. population and RIAS–2 standardization sample: age and gender�� 121 Comparison of U.S. population and RIAS-2 standardization sample: age and ethnicity�� 122 T-score conversions for subtest raw scores, ages 8:0–8:3�� 123 Standard score conversions for the RIAS-2 Verbal Intelligence Index�� 125 Table 3.5 Table 3.6 Table 3.7 Table 3.8 Table 4.1 Table 4.2 Table 4.3 Table 4.4 Table 4.5 Table 4.6 Major types of reliability�� 142 Half-test coefficients and corresponding full-test coefficients corrected with the Spearman-Brown formula�� 146 Calculating KR-20�� 148 Calculating coefficient alpha�� 149 Calculating inter-rater agreement�� 150 Calculating the reliability of a linear combination�� 153 xix xx List of Tables Table 4.7 Table 4.8 Calculating the reliability of difference scores�� 154 Major sources of error variance associated with types of reliability�� 155 Table 4.9 Reliability expected when increasing the number of items�� 160 Table 4.10 Applying a correction for attenuation�� 164 Table 4.11 Standard errors of measurement for different levels of reliability and standard deviations�� 166 Table 4.12 Coefficient alpha for RIAS–2 subtests�� 174 Table 4.13 Coefficient alpha for RIAS–2 Indexes by age group�� 174 Table 4.14 Standard errors of measurement of the RIAS–2 indexes by age group�� 175 Table 4.15 Stability coefficients of scores for the RIAS–2 subtests and indexes�� 176 Table 4.16 KR-21 reliability estimates for tests with a mean of 80%�� 178 Table 5.1 Table 5.2 Table 5.3 Table 5.4 Table 5.5 Table 5.6 Table 5.7 Table 5.8 Tracing historical trends in the concept of validity�� 193 Correlation matrix for a hypothetical intelligence test�� 200 Factor matrix for a hypothetical intelligence test�� 202 Rotated factor matrix for a hypothetical intelligence test�� 202 Calculating sensitivity and specificity�� 210 A multitrait-multimethod matrix�� 212 Sources of validity evidence�� 215 RIAS–2 intelligence subtest rotated loadings for individuals 18–30 years of age�� 217 Table 6.1 Table 6.2 Table 6.3 Table 6.4 Table 6.5 Table 6.6 Table 6.7 Table 6.8 Table 6.9 Strengths and weaknesses of selected-response items�� 230 Strengths and weaknesses of constructed-response items�� 231 General item development guidelines�� 235 Checklist for the development of multiple-choice items�� 242 Checklist for the development of true–false items�� 243 Checklist for the development of matching items�� 245 Checklist for the development of essay items�� 248 Checklist for the development of short-answer items�� 249 Checklist for the development of typical-response items�� 259 Table 7.1 Table 7.2 Table 7.3 Optimal p values for items with varying numbers of choices�� 267 Guidelines for evaluating D values�� 270 Maximum D values at different difficulty levels�� 271 Table 8.1 Table 8.2 Table 8.3 Table 8.4 Major publishers of standardized group achievement tests�� 303 State assessment practices (2008)�� 308 Important test-taking skills to teach students�� 313 Sample of achievement test applications outside of public schools��327 Table 9.1 Table 9.2 Organization of major group intelligence/aptitude tests�� 351 Organization of the Wechsler Intelligence Scale for Children-Fifth Edition (WISC-V)�� 356 Organization of the Stanford-Binet Intelligence Scales, 5th Edition (SB5)�� 358 Table 9.3 List of Tables Table 9.4 Table 9.5 xxi Organization of the Woodcock-Johnson IV (WJ IV) Tests of Cognitive Abilities�� 363 Organization of the Reynolds Intellectual Assessment Scales, Second Edition�� 364 Table 10.1 MMPI-2 Clinical scales�� 399 Table 10.2 Original MMPI-2 validity scales�� 401 Table 10.3 Domain and Facet scales for the NEO PI-3�� 410 Table 10.4 MCMI-IV scales�� 413 Table 11.1 Ten most popular tests of child personality and behavior�� 434 Table 11.2 BASC-3 composites, primary scales, and content scales in the TRS and PRS�� 439 Table 11.3 Clinical Assessment Scales for the Elderly—clinical scales and descriptions�� 449 Table 11.4 Behavioral categories of the BASC-3 student observation system (SOS)�� 452 Table 12.1 Steps for developing effective structured interviews�� 469 Table 12.2 Big 5 personality dimensions�� 471 Table 12.3 Summary of contemporary views regarding personality testing in job settings�� 473 Table 12.4 Dimensions from the Personnel Selection Inventory�� 476 Table 12.5 Examples of dimensions assessed on biodata questionnaires�� 479 Table 12.6 Summary of advantages and disadvantages of personnel selection approaches�� 480 Table 12.7 Common error types when evaluating job performance�� 487 Table 12.8 Key Concepts from the Uniform Guidelines on Employee Selection Procedures�� 489 Table 12.9 Key Points from the Principles for the Validation and Use of Personnel Selection Procedures�� 490 Table 12.10 Holland’s RIASEC personality type categories�� 491 Table 13.1 Halstead-Reitan Neuropsychological Test Battery�� 508 Table 13.2 Neuropsychological instruments commonly employed in a flexible battery�� 511 Table 13.3 Symptom presentations in subtypes of dementia�� 514 Table 13.4 Core and supplementary subtests and indexes available for the TOMAL-2�� 516 Table 13.5 Description of TOMAL-2 subtests�� 517 Table 13.6 Test results showing medication effects on neuropsychological test performance�� 530 Table 13.7 Determining change in test scores using the Reliable Change Index (RCI)�� 535 Table 14.1 Examples of types of competency evaluations where psychological tests are often used�� 551 Table 14.2 Summary of factors to consider under Daubert (1993) for admissibility of expert testimony in federal (and many state) courts�� 564 xxii List of Tables Table 16.1 Accommodations involving modifications of presentation format�� 622 Table 16.2 Accommodations involving modifications of response format�� 623 Table 16.3 Accom- modations involving modifications of timing�� 624 Table 16.4 Accom-modations involving modifications of setting�� 625 Table 16.5 Accommodations involving adaptive devices and supports�� 626 Table 16.6 Determining which accommodations to provide�� 632 Table 17.1 Checklist for developing assessments�� 645 Table 17.2 Checklist for selecting published assessments�� 648 Table 17.3 Checklist for administering assessments�� 652 Table 17.4 Checklist for scoring assessments�� 653 Table 17.5 Checklist for interpreting, using, and communicating assessment results�� 657 Table 17.6 Responsibilities of test takers�� 660 Table 18.1 Phase one: steps in test conceptualization�� 665 Table 18.2 Example of a test manual’s user qualifications section�� 668 Table 18.3 Phase II: specification of test structure and format�� 675 Table 18.4 Table of specifications for test on a chapter on basic statistics (number of items by content area and level of objective)�� 679 Table 18.5 Table of specifications for a measure of depression�� 679 Table 18.6 Phase III: planning, standardization, and psychometric studies�� 684 Table A.1 Proportions of area under the normal curve�� 693 List of Special Interest Topics 1.1: How Do Clinical, Counseling, and School Psychologists Differ?�� 3 1.2: Making the Test�� 35 1.3: What Does the Twenty-First Century Hold for the Assessment Profession?�� 42 2.1: 2.2: 2.3: 2.4: 2.5: 2.6: Population Parameters and Sample Statistics�� 61 A Public Outrage: Physicians Overcharge Their Patients�� 66 Is the Variance Always Larger Than the Standard Deviation?�� 69 Whence the Normal Curve?�� 70 Are Weak Correlations Useless or of No Practical Value?�� 74 Caution: Drawing Conclusions of Causality�� 82 3.1: 3.2: 3.3: 3.4: The “Flynn Effect”�� 96 Using Clinical Norms?�� 98 Why Do IQ Tests Use a Mean of 100 and Standard Deviation of 15?�� 103 The History of Stanine Scores�� 106 4.1: 4.2: 4.3: 4.4: The Reliability Index�� 141 Consistency of Classification with Mastery Tests�� 162 Asymmetrical Confidence Intervals�� 168 A Quick Way to Estimate Reliability for Classroom Exams�� 176 5.1: Are Psychological Tests as Accurate as Medical Tests?�� 188 6.1: 6.2: 6.3: 6.4: Do Multiple-Choice Items Penalize Creative Examinees?�� 228 What Research Says About “Changing your answer?”�� 242 Mixing Item Formats�� 252 Guttman Scales and Thurstone Scales�� 254 7.1: Item Difficulty Indexes and Power Tests�� 268 7.2: Item Analysis for Constructed-Response Items�� 271 8.1: 8.2: 8.3: 8.4: 8.5: The “Nation’s Report Card”�� 295 Federal Law and Achievement Assessment�� 297 Finding Information on Standardized Tests�� 304 Standards-Based Assessments�� 306 American Educational Research Association (AERA) Position Statement on High-Stakes Testing�� 309 xxiii xxiv List of Special Interest Topics 8.6: Value-Added Assessment: A New Approach to Educational Accountability�� 310 8.7: Administering Standardized Group Achievement Tests�� 315 8.8: A Brief History of Grading�� 323 9.1: Do Intelligence Tests Reveal Your Potential for Learning or Other Achievements?�� 335 9.2: The Controversial IQ: Knowns and Unknowns�� 337 9.3: The Controversial IQ: Schools and IQ Tests�� 338 9.4: Ability Profiles on the CogAT�� 350 9.5: The Cattell-Horn-Carroll (CHC) Theory of Intelligence and Its Impact on Contemporary Intelligence Test Batteries�� 358 9.6: Example of a Computerized Report of an Intellectual Assessment of an Adult�� 366 10.1: The Handsome and the Deformed Leg�� 385 10.2: Therapeutic Assessment�� 387 10.3: An Example of a “Fake Good” Response Set�� 392 10.4: The Forer Effect—or, You and Your Aunt Fanny!�� 394 10.5: The MMPI-3: The Next Step in the Evolution of an Instrument�� 402 11.1: An Example of a “Fake Bad” Response Set on a Parent Behavior Rating Scale�� 432 11.2: Categorical Versus Dimensional Diagnosis�� 443 12.1: Contributions of Female Psychologists in Early I/O Psychology�� 464 13.1: Common Brain Injury Referrals�� 520 13.2: Anosognosia�� 524 13.3: How Does a Neuropsychologist Measure Effort?�� 527 13.4: Case Study of Medication Effects on Neuropsychological Testing�� 529 13.5: How Does a Neuropsychologist Measure Change?�� 533 14.1: Why I Use Psychological and Neuropsychological Tests in Criminal Forensic Practice�� 546 14.2: The Flynn Correction in Death Penalty Litigation: An Advocacy Perspective�� 557 14.3: Use of the BASC-3 in Custody Evaluations�� 561 14.4: Third Party Observers: Why You Ought Not to Watch�� 565 15.1: Sex Differences in Intelligence�� 575 15.2: Courtroom Controversy Over IQ Testing in the Public Schools�� 580 15.3: Fairness and Bias: A Complex Relationship�� 584 15.4: Stereotype Threat: An Emerging But Controversial Explanation of Group Differences on Various Tests of Mental Abilities�� 590 15.5: Why Is the Cultural Test Bias Hypothesis So Robust in the Profession: And Why Do So Many Hold the Mean Differences Equals Bias Ideology?�� 592 List of Special Interest Topics xxv 16.1: Major Legislation That Guides the Assessment of Students with Disabilities�� 618 16.2: Understanding the Current State of Test Accommodation Implementation�� 627 16.3: Allowable Accommodations in a Statewide Curriculum and Assessment Program�� 631 16.4: Assessment of Students with Disabilities: Selected Legal Issues�� 635 17.1: Wikipedia and the Rorschach Ink Blot Test�� 649 17.2: Aspects of Informed Consent�� 650 17.3: The Accuracy of Clinical Versus Actuarial Approach to Interpreting Assessment Results�� 655 17.4: Steps to Prevent Student Cheating�� 659 17.5: Teachers Cheating?�� 659 18.1: Bloom’s Taxonomy of Cognitive Objectives�� 677

Mastering Modern Psychological Testing: Theory & Methods

Related documents

Products

Support

Mastering Modern Psychological Testing: Theory & Methods

Related documents

Add this document to collection(s)

Add this document to saved

Suggest us how to improve StudyLib