Measuring Achievement and Aptitude: Applications for Counseling Session 7 Definitions Achievement tests – Provides information about what an individual has learned or acquired. Provide information to help individuals understand their academic strengths and limitations. Aptitude test – Predict future performance or ability to learn Often come to counseling because they are trying to make a decision about their future. Six Areas of Assessment Using Achievement Survey achievement batteries Individual achievement tests – Typically cover the areas of reading, math, and spelling Diagnostic tests – Diagnosis learning disabilities – Assess achievement strengths and limitations Six Areas--2 Criterion-referenced tests – Measure knowledge or comprehension to determine if certain criterion or standard has been met Minimum-level skills – Measure skills for promotion, entrance, or graduation – Level is established before test is administered Subject area tests – Example: test that covered knowledge of assessment strategies in counseling Achievement Battery TerraNova TerraNova – – – – – – Administered to children K – 12. Combination of norm and criterion referenced Schools can select from basic battery Schools can select format Spanish version Construction involved Item Response Theory – – – – Norm-referenced Criterion-referenced Objective master Performance level – – – – National percentiles National percentile ranges Stanines Grade equivalents – – – Reliability r= mid .80 to mid .90 Recently published not much on construct validity yet Content validity Information Provided Provides a profile Scores provided Psychometric properties Item analysis Discussion Would you use this test? Provide rationale? What do others think ? Aptitude Assessments Hood & Johnson (1997) argued that counselors in a variety of fields need to be knowledgeable about the predominant scholastic aptitudes tests. If you do not have a working knowledge of these instruments may not be viewed as credible. SAT Scholastic Assessment Test – First adminstered in 1926 as the Scholastic Aptitude Tests – 1994 was revised and renamed SAT – 2004 added an essay—writing sample Consists of two tests – SAT I: Reasoning test 3 hour test that measures verbal and mathematical reasoning abilities – SAT II: Subject tests Consists of single subject tests – – – – Writing Math level 1 Biology French Interpretation Score ranges between 200 and 800 on both the Verbal and Mathematical sections of SAT I – Mean is 500 – SD is 100 ETS (Education Testing Services) uses complex formula to equate scores on different versions. Psychometric Properties of Aptitude Tests These tests are used to make decisions that often have a significant influence on people’s lives and, therefore, the validity of these instruments deserves analysis. GRE – Combined scores have validity coefficients of .31-.37 – Undergraduate grade point averages have r = .35-.39 – Adding the GRE and Grade point results in r = .49 -.63 Discussion Would you use this test? Provide What rationale? do others think? Work Samples Assessment Another way of assessing vocational/career aptitudes Philosophical base – Work performance can best be assessed by using a sample of the actual work the individual would perform Work Sample Assessments Valpar Component Work Sample (VCWS) – 23 individual work samples on computer – Criterion-referenced scoring – Speed test (completed in certain amount of time_ – Norms on nondisabled and disabled workers – Comes with built in computer scoring system SAGE system – Non computer and computer versions Test Preparation and Performance Do workshops really increase scores? – Mixed review – Depends on individuals test taking sophistication—can they learn to learn to meet exams requirement or test format (logical problem solving) – Individual who have experience in taking standardized test have a distinct advantage (Anastasi, 1981) – Manuals and – sample tests have been constructed to level the playing field and provide persons with such experiences. Coaching – Coaching programs do make a significant positive difference in scores (contested but not yet disproven) – Less expensive ways (travel experiences, tutoring, trips to museums – The closer the coaching material to the actual test content the greater the improvement in test scores – The more time individuals spend reviewing the material, the more likely it is that they will cover the material on the test. Exam Results 2 1 0 92.25 106.5 108 116.25 Mean = 122.3 s.d. = 17.08 Mode = {106,5; 108; 150} Median = 118.5 117 118.5 126 129 130.5 133.5 142.5 Z= data point – mean standard deviation 150 Raw and Z-scores Raw scores 92.25 106.5 106.5 108 108 116.25 117 118.5 126 129 130.5 133.5 142.5 150 150 Mean = 122.3 s.d. = 17.08 Z-scores -1.75934498 -0.925046612 -0.925046612 -0.837225731 -0.837225731 -0.354210886 -0.310300446 -0.222479565 0.216624839 0.392266601 0.480087482 0.655729244 1.182654529 1.621758933 1.621758933